Load control system having a visible light sensor

ABSTRACT

A visible light sensor (VLS) may be configured to sense environmental characteristics using images of a space. The VLS may be controlled in one or more modes, including a daylight glare sensor mode, a daylighting sensor mode, a color sensor mode, and/or an occupancy/vacancy sensor mode. In the daylight glare sensor mode, the VLS may be configured to decrease or eliminate glares. In the daylighting sensor mode and/or color sensor mode, the VLS may be configured to provide a preferred amount of light and color temperature, respectively, within the space. In the occupancy/vacancy sensor mode, the VLS may be configured to detect an occupancy/vacancy condition within the space and adjust one or more control devices according to the occupation or vacancy of the space. The visible light sensor may be configured to protect the privacy of users within the space via software, a removable module, and/or a special sensor.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a continuation of U.S. Non-Provisional applicationSer. No. 16/857,267, filed Apr. 24, 2020, which is a continuation ofU.S. Non-Provisional application Ser. No. 15/838,263, filed Dec. 11,2017, which issued as U.S. Pat. No. 10,660,185 on May 19, 2020, whichclaims priority from U.S. Provisional Patent Application No. 62/432,477,filed Dec. 9, 2016 all of which are hereby incorporated by reference intheir entireties.

BACKGROUND

A user environment, such as a residence or an office building, forexample, may be configured using various types of load control systems.A lighting control system may be used to control the lighting loadsproviding artificial light in the user environment. A motorized windowtreatment control system may be used to control the natural lightprovided to the user environment. An HVAC system may be used to controlthe temperature in the user environment.

Each load control system may include various control devices, includinginput devices and load control devices. The load control devices mayreceive digital messages, which may include load control instructions,for controlling an electrical load from one or more of the inputdevices. The load control devices may be capable of directly controllingan electrical load. The input devices may be capable of indirectlycontrolling the electrical load via the load control device.

Examples of load control devices may include lighting control devices(e.g., a dimmer switch, an electronic switch, a ballast, or alight-emitting diode (LED) driver), a motorized window treatment, atemperature control device (e.g., a thermostat), an AC plug-in loadcontrol device, and/or the like. Examples of input devices may includeremote control devices, occupancy sensors, daylight sensors, glaresensors, color temperature sensors, temperature sensors, and/or thelike. Remote control devices may receive user input for performing loadcontrol. Occupancy sensors may include infrared (IR) sensors fordetecting occupancy/vacancy of a space based on movement of the users.Daylight sensors may detect a daylight level received within a space.Glare sensors may be positioned facing outside of a building (e.g., on awindow or exterior of a building) to identify the position of the sunwhen in view of the glare sensor. Color temperature sensors determinethe color temperature within a user environment based on the wavelengthsand/or frequencies of light. Temperature sensors may detect the currenttemperature of the space.

As described herein, current load control systems implement many inputdevices, including a number of different sensors. The use of many inputdevices causes the load control systems to take readings from multipledifferent types of devices and control loads based on many differenttypes of input.

The input devices in current load control systems may also beinefficient for performing their independent functions in the loadcontrol systems. For example, current load control systems may receiveinput from a glare sensor that indicates that glare is being receivedfrom the sun, but load control systems may attempt to reduce oreliminate the amount of glare within the user environment usingprediction algorithms to predict the portions of the user environmentthat are being affected by glare. Attempting to reduce or eliminate theamount of glare within the user environment using these predictionalgorithms may be unreliable.

The daylight sensors and the color temperature sensors in the loadcontrol systems may also be inefficient for gathering accurateinformation for performing load control. Current use of daylight sensorsand color temperature sensors rely on the accuracy of the location ofthe sensor for detecting how the intensity of light affects the userenvironment. It may be desirable to have more accurate ways ofdetermining how the actual intensity and color of light provided withinthe user environment affects a user within the environment.

As the occupancy/vacancy sensor generally senses the presence or absenceof a person within the user environment using passive infra-red (PIR)technology, the occupancy/vacancy sensor may fail to detect theoccupancy of a room due to the lack of movement by a user. Theoccupancy/vacancy sensor senses the presence of a person using the heatmovement of the person. The vacancy sensor determines a vacancycondition within the user environment in the absence of the heatmovement of a person for a specified timeout period. Theoccupancy/vacancy sensor may detect the presence or absence of a userwithin the user environment, but the sensor may fail to provide accurateresults. For example, the occupancy/vacancy sensor may detect other heatsources within a user environment and inaccurately determine that theheat sources are emanating from a person. Further, the occupancy/vacancysensor is unable to identify a person that is not moving, or that ismaking minor movements, within the user environment. Thus, it may bedesirable to otherwise determine occupancy/vacancy within a userenvironment.

As complex load control systems generally include many different typesof input devices for gathering information about a load controlenvironment, the processing and communicating of information in suchsystems can be inefficient. Additionally, as the information collectedby many input devices may be inaccurate, the control of loads accordingto such information may also be inaccurate.

SUMMARY

The present disclosure relates to a load control system for controllingthe amount of power delivered to one or more electrical load, and moreparticularly, to a load control system having a visible light sensor fordetecting occupancy and/or vacancy conditions in a space.

As described herein, a sensor for sensing environmental characteristicsof a space comprises a visible light sensing circuit configured torecord an image of the space and a control circuit responsive to thevisible light sensing circuit. The control circuit may be configured todetect at least one of an occupancy condition and a vacancy condition inthe space in response to the visible light sensing circuit, and tomeasure a light level in the space in response to the visible lightsensing circuit.

The visible light sensor may perform differently depending on the modein which the visible light sensor is operating. For example, the visiblelight sensor may detect and/or adjust an environmental characteristicwithin a space based on the mode in which the visible light sensor isoperating. The visible light sensor may operate in a particular mode fora period of time and/or the visible light sensor may switch from onemode to another mode after the same, or different, period of time. Themodes in which the visible light sensor may operate may include asunlight glare sensor mode, a daylighting sensor mode, a colortemperature sensor mode, an occupancy/vacancy sensor mode, etc.

The control circuit may be configured to detect a first environmentalcharacteristic of the space by applying a first mask to focus on a firstregion of interest of the image, and to detect a second environmentalcharacteristic of the space by applying a second mask to focus on asecond region of interest of the image. The control circuit may beconfigured to apply the first mask to focus on the first region ofinterest of the image in order to detect at least one of an occupancycondition and a vacancy condition in the space. The control circuit maybe configured to apply the second mask to focus on the second region ofinterest of the image in order to measure a light level in the space.

The control circuit may be configured to perform a number of sequentialsensor events for sensing a plurality of environmental characteristicsin response to the image. Each sensor event may be characterized by oneof the plurality of environmental characteristics to detect during thesensor event and a respective mask. The control circuit may beconfigured to perform one of the sensor events to detect the respectiveenvironmental characteristic by applying the respective mask to theimage to focus on a region of interest and process the portion of theimage in the region of interested using to a predetermined algorithm forsensing the respective environmental characteristic.

The visible light sensor may record an image of the and process theimage to determine a location of at least one object within the space.Based on the location of the at least one object within the space, anoperation of the sensor may be automatically configured in response toan environmental characteristic of the space identified in the image.

Digital masks may be automatically configured for identifying anunmasked portion of the image to be processed to sense the environmentalcharacteristics of the space. Movement and/or objects may be detected inthe unmasked portions of the space. Different sensitivity levels may beconfigured for detecting movement in the unmasked portions of the space.Transparent objects may be masked to avoid detection of movement throughthe transparent objects.

The operation of the sensor may be configured during a configurationprocedure or dynamically during operation after the configurationprocedure. The dynamic configuration may be in response to a movement ofa location of an object within the space.

The sensor may be automatically configured to control a lighting loadusing sensor modes and at least one control parameter during operation.The sensor mode may be at least one of a daylighting sensor mode, adaylight glare sensor mode, a color sensor mode, an occupancy/vacancysensor mode, or an occupant count mode. The at least one controlparameter may be at least one of a light sensitivity, a light gain, or acolor temperature.

Automatic configuration of zones may be performed based on the images ofthe space. A lighting zone may be automatically created for controllinglighting devices in the lighting zone. The lighting zone may be afunctional lighting zone or a decorative lighting zone. For example, thelighting zone may be a daylighting zone based on the identification oflighting fixtures in an image relative to a window. The lighting zonemay include decorative lighting fixtures identified within the image.The lighting zone may be identified based on the location of identifiedlighting fixtures and/or the location of illuminated portions of thespace.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a diagram of an example load control system having a visiblelight sensor.

FIGS. 2A-2G show example images of a room that may be recorded by acamera of a visible light sensor.

FIG. 3 is a block diagram of an example visible light sensor.

FIGS. 4A and 4B are sequence diagrams for controlling load controldevices based on images captured by a visible light sensor.

FIG. 5 shows a flowchart of an example sensor event procedure that maybe executed by a visible light sensor.

FIG. 6 shows a flowchart of an example occupancy/vacancy detectionprocedure that may be executed by a visible light sensor.

FIG. 7 shows a flowchart of an example vacancy time procedure that maybe executed by a visible light sensor.

FIG. 8 shows a flowchart of an example baseline configuration procedurefor generating and storing baseline images.

FIG. 9A shows a flowchart of an example procedure for determining theimpact of light emitted by lighting fixtures on sub-areas of a space.

FIGS. 9B-9E show example nighttime images of a room with a mask beingapplied.

FIG. 10A shows a flowchart of an example procedure for measuring andcontrolling lighting levels on a task area or other region of interest.

FIGS. 10B-10D show example images that illustrate the subtraction andbackfill process.

FIGS. 10E-10G show example images that illustrate the baseline process.

FIG. 11 shows a flowchart of another example procedure for measuring andcontrolling a lighting level on a task area or other region of interest.

FIGS. 12A and 12B show a flowchart of an example procedure forcontrolling lighting fixtures to provide a uniform predefined lightprofile on a task area or other region of interest.

FIG. 13 shows a flowchart of an example baseline configuration procedurethat may be executed by a visible light sensor and/or a systemcontroller.

FIG. 14 shows a flowchart of an example procedure for controlling acorrelated color temperature (CTT) value based on an image.

FIG. 15 shows a flowchart of example glare detection and controlprocedure.

FIG. 16 shows a flowchart of another example glare detection and controlprocedure.

FIG. 17 shows a flowchart of an example configuration procedure that maybe executed to configure a visible light sensor and/or a systemcontroller for operation.

FIG. 18 shows a flowchart of another example configuration procedurethat may be executed to configure a visible light sensor and/or a systemcontroller for operation.

FIG. 19 shows a flowchart of an example configuration procedure that maybe executed to automatically configure a visible light sensor and/or asystem controller for operation.

FIG. 20 shows a flowchart of an example zone configuration procedurethat may be executed to configure one or more zones within a space.

FIG. 21 is a block diagram illustrating an example network device.

FIG. 22 is a block diagram illustrating an example system controller.

FIG. 23 is a block diagram illustrating an example control-targetdevice.

DETAILED DESCRIPTION

FIG. 1 is a simple diagram of an example load control system 100 forcontrolling the amount of power delivered from an alternating-current(AC) power source (not shown) to one or more electrical loads. The loadcontrol system 100 may be installed in a room 102 of a building. Theload control system 100 may comprise a plurality of control devicesconfigured to communicate with each other via wireless signals, e.g.,radio-frequency (RF) signals 108. Alternatively or additionally, theload control system 100 may comprise a wired digital communication linkcoupled to one or more of the control devices to provide forcommunication between the load control devices. The control devices ofthe load control system 100 may comprise a number of control-sourcedevices (e.g., input devices operable to transmit digital messages inresponse to user inputs, occupancy/vacancy conditions, changes inmeasured lighting intensity, etc.) and a number of control-targetdevices (e.g., load control devices operable to receive digital messagesand control respective electrical loads in response to the receiveddigital messages). A single control device of the load control system100 may operate as both a control-source and a control-target device.

The control-source devices may be configured to transmit digitalmessages directly to the control-target devices. In addition, the loadcontrol system 100 may comprise a system controller 110 (e.g., a centralprocessor or load controller) operable to communicate digital messagesto and from the control devices (e.g., the control-source devices and/orthe control-target devices). For example, the system controller 110 maybe configured to receive digital messages from the control-sourcedevices and transmit digital messages to the control-target devices inresponse to the digital messages received from the control-sourcedevices. The control-source and control-target devices and the systemcontroller 110 may be configured to transmit and receive the RF signals108 using a proprietary RF protocol, such as the ClearConnect® protocol.Alternatively, the RF signals 108 may be transmitted using a differentRF protocol, such as, a standard protocol, for example, one of WIFI,ZIGBEE, Z-WAVE, KNX-RF, ENOCEAN RADIO protocols, or a differentproprietary protocol.

The load control system 100 may comprise one or more load controldevices, e.g., a lighting control device (e.g., dimmer switch, LEDdriver, ballast, etc.) for controlling for controlling one or more oflighting fixtures 172, 174, 176, 178. Each of the lighting fixtures 172,174, 176, 178 may comprise a lighting load (e.g., a light-emitting diode(LED) light source) and a respective lighting control device (e.g., anLED driver) for controlling the lighting load of the lighting fixture.

The lighting control devices (e.g., the LED drivers for the lightingfixtures 172, 174, 176, 178) may be configured to wirelessly receivedigital messages via the RF signals 108 (e.g., from the systemcontroller 110) and to control the lighting load 122 in response to thereceived digital messages. Examples of lighting control devices operableto transmit and receive digital messages is described in greater detailin commonly-assigned U.S. Patent Application Publication No.2009/0206983, published Aug. 20, 2009, entitled COMMUNICATION SYSTEM FORA RADIO-FREQUENCY LOAD CONTROL SYSTEM, the entire disclosure of which ishereby incorporated by reference.

The lighting control devices (e.g., the LED drivers for the lightingfixtures 172, 174, 176, 178) may receive instructions for controllingthe color temperature of the corresponding lighting loads. Examples ofLED drivers configured to control the color temperature of LED lightsources are described in greater detail in commonly-assigned U.S. PatentApplication Publication No. 2014/0312777, published Oct. 23, 2014,entitled SYSTEMS AND METHODS FOR CONTROLLING COLOR TEMPERATURE, theentire disclosure of which is hereby incorporated by reference. The loadcontrol system 100 may further comprise other types of remotely-locatedload control devices, such as, for example, electronic dimming ballastsfor driving fluorescent lamps.

The load control system 100 may comprise a plug-in load control device140 for controlling a plug-in electrical load, e.g., a plug-in lightingload (such as a floor lamp 142 or a table lamp) and/or an appliance(such as a television or a computer monitor). For example, the floorlamp 142 may be plugged into the plug-in load control device 140. Theplug-in load control device 140 may be plugged into a standardelectrical outlet 144 and thus may be coupled in series between the ACpower source and the plug-in lighting load. The plug-in load controldevice 140 may be configured to receive digital messages via the RFsignals 108 (e.g., from the system controller 110) and to turn on andoff or adjust the intensity of the floor lamp 142 in response to thereceived digital messages.

Alternatively or additionally, the load control system 100 may comprisecontrollable receptacles for controlling plug-in electrical loadsplugged into the receptacles. The load control system 100 may compriseone or more load control devices or appliances that are able to directlyreceive the wireless signals 108 from the system controller 110, such asa speaker 146 (e.g., part of an audio/visual or intercom system), whichis able to generate audible sounds, such as alarms, music, intercomfunctionality, etc.

The load control system 100 may comprise one or more daylight controldevices, e.g., motorized window treatments 150, such as motorizedcellular shades, for controlling the amount of daylight entering theroom 102. Each motorized window treatment 150 may comprise a windowtreatment fabric 152 hanging from a headrail 154 in front of arespective window 104. Each motorized window treatment 150 may furthercomprise a motor drive unit (not shown) located inside of the headrail154 for raising and lowering the window treatment fabric 152 forcontrolling the amount of daylight entering the room 102. The motordrive units of the motorized window treatments 150 may be configured toreceive digital messages via the RF signals 108 (e.g., from the systemcontroller 110) and adjust the position of the respective windowtreatment fabric 152 in response to the received digital messages. Theload control system 100 may comprise other types of daylight controldevices, such as, for example, a cellular shade, a drapery, a Romanshade, a Venetian blind, a Persian blind, a pleated blind, a tensionedroller shade systems, an electrochromic or smart window, and/or othersuitable daylight control device. Examples of battery-powered motorizedwindow treatments are described in greater detail in U.S. Pat. No.8,950,461, issued Feb. 10, 2015, entitled MOTORIZED WINDOW TREATMENT,and U.S. Patent Application Publication No. 2014/0305602, published Oct.16, 2014, entitled INTEGRATED ACCESSIBLE BATTERY COMPARTMENT FORMOTORIZED WINDOW TREATMENT, the entire disclosures of which are herebyincorporated by reference.

The load control system 100 may comprise one or more temperature controldevices, e.g., a thermostat 160 for controlling a room temperature inthe room 102. The thermostat 160 may be coupled to a heating,ventilation, and air conditioning (HVAC) system 162 via a control link(e.g., an analog control link or a wired digital communication link).The thermostat 160 may be configured to wirelessly communicate digitalmessages with a controller of the HVAC system 162. The thermostat 160may comprise a temperature sensor for measuring the room temperature ofthe room 102 and may control the HVAC system 162 to adjust thetemperature in the room to a setpoint temperature. The load controlsystem 100 may comprise one or more wireless temperature sensors (notshown) located in the room 102 for measuring the room temperatures. TheHVAC system 162 may be configure to turn a compressor on and off forcooling the room 102 and to turn a heating source on and off for heatingthe rooms in response to the control signals received from thethermostat 160. The HVAC system 162 may be configured to turn a fan ofthe HVAC system on and off in response to the control signals receivedfrom the thermostat 160. The thermostat 160 and/or the HVAC system 162may be configured to control one or more controllable dampers to controlthe air flow in the room 102. The thermostat 160 may be configured toreceive digital messages via the RF signals 108 (e.g., from the systemcontroller 110) and adjust heating, ventilation, and cooling in responseto the received digital messages.

The load control system 100 may comprise one or more other types of loadcontrol devices, such as, for example, a screw-in luminaire including adimmer circuit and an incandescent or halogen lamp; a screw-in luminaireincluding a ballast and a compact fluorescent lamp; a screw-in luminaireincluding an LED driver and an LED light source; an electronic switch,controllable circuit breaker, or other switching device for turning anappliance on and off; a controllable electrical receptacle orcontrollable power strip for controlling one or more plug-in loads; amotor control unit for controlling a motor load, such as a ceiling fanor an exhaust fan; a drive unit for controlling a projection screen;motorized interior or exterior shutters; an air conditioner; acompressor; an electric baseboard heater controller; a variable airvolume controller; a fresh air intake controller; a ventilationcontroller; hydraulic valves for use with radiators and radiant heatingsystems; a humidity control unit; a humidifier; a dehumidifier; a waterheater; a boiler controller; a pool pump; a refrigerator; a freezer; atelevision or computer monitor; a video camera; an audio system oramplifier; an elevator; a power supply; a generator; an electriccharger, such as an electric vehicle charger; an alternative energycontroller; and/or another load control device.

The load control system 100 may comprise one or more input devices,e.g., such as a remote control device 170 and/or a visible light sensor180. The input devices may be fixed or movable input devices. The systemcontroller 110 may be configured to transmit one or more digitalmessages to the load control devices (e.g., a lighting control device ofthe lighting fixtures 172, 174, 176, 178, the plug-in load controldevice 140, the motorized window treatments 150, and/or the thermostat160) in response to the digital messages received from the remotecontrol device 170 and/or the visible light sensor 180. The remotecontrol device 170 and/or the visible light sensor 180 may be configuredto transmit digital messages directly to the lighting control device ofthe lighting fixtures 172, 174, 176, 178, the plug-in load controldevice 140, the motorized window treatments 150, and/or the temperaturecontrol device 160.

The remote control device 170 may be configured to transmit digitalmessages via the RF signals 108 to the system controller 110 (e.g.,directly to the system controller 110) in response to an actuation ofone or more buttons of the remote control device 170. For example, theremote control device 170 may be battery-powered. The load controlsystem 100 may comprise other types of input devices, such as, forexample, temperature sensors, humidity sensors, radiometers, cloudy-daysensors, shadow sensors, pressure sensors, smoke detectors, carbonmonoxide detectors, air-quality sensors, motion sensors, securitysensors, proximity sensors, fixture sensors, partition sensors, keypads,multi-zone control units, slider control units, kinetic or solar-poweredremote controls, key fobs, cell phones, smart phones, tablets, personaldigital assistants, personal computers, laptops, timeclocks,audio-visual controls, safety devices, power monitoring devices (e.g.,such as power meters, energy meters, utility submeters, utility ratemeters, etc.), central control transmitters, residential controllers,commercial controllers, industrial controllers, and/or any combinationthereof.

The system controller 110 may be coupled to a network, such as awireless or wired local area network (LAN), e.g., for access to theInternet. The system controller 110 may be wirelessly connected to thenetwork, e.g., using Wi-Fi technology. The system controller 110 may becoupled to the network via a network communication bus (e.g., anEthernet communication link). The system controller 110 may beconfigured to communicate via the network with one or more networkdevices, e.g., a mobile device 190, such as, a personal computing deviceand/or a wearable wireless device. The mobile device 190 may be locatedon an occupant 192, for example, may be attached to the occupant's bodyor clothing or may be held by the occupant. The mobile device 190 may becharacterized by a unique identifier (e.g., a serial number or addressstored in memory) that uniquely identifies the mobile device 190 andthus the occupant 192. Examples of personal computing devices mayinclude a smart phone (for example, an iPhone® smart phone, an Android®smart phone, or a Blackberry® smart phone), a laptop, and/or a tabletdevice (for example, an iPad® hand-held computing device). Examples ofwearable wireless devices may include an activity tracking device (suchas a FitBit® device, a Misfit® device, and/or a Sony Smartband® device),a smart watch, smart clothing (e.g., OMsignal® smartwear, etc.), and/orsmart glasses (such as Google Glass® eyewear). In addition, the systemcontroller 110 may be configured to communicate via the network with oneor more other control systems (e.g., a building management system, asecurity system, etc.).

The mobile device 190 may be configured to transmit digital messages tothe system controller 110, for example, in one or more Internet Protocolpackets. For example, the mobile device 190 may be configured totransmit digital messages to the system controller 110 over the LANand/or via the internet. The mobile device 190 may be configured totransmit digital messages over the internet to an external service(e.g., If This Then That (IFTTT®) service), and then the digitalmessages may be received by the system controller 110. The mobile device190 may transmit and receive RF signals 109 via a Wi-Fi communicationlink, a Wi-MAX communications link, a Bluetooth communications link, anear field communication (NFC) link, a cellular communications link, atelevision white space (TVWS) communication link, or any combinationthereof to communicate with the system controller, for example.Alternatively, or additionally, the mobile device 190 may be configuredto transmit RF signals according to a proprietary protocol. The loadcontrol system 100 may comprise other types of network devices coupledto the network, such as a desktop personal computer, a Wi-Fi orwireless-communication-capable television, or any other suitableInternet-Protocol-enabled device. Examples of load control systemsoperable to communicate with mobile and/or network devices on a networkare described in greater detail in commonly-assigned U.S. PatentApplication Publication No. 2013/0030589, published Jan. 31, 2013,entitled LOAD CONTROL DEVICE HAVING INTERNET CONNECTIVITY, the entiredisclosure of which is hereby incorporated by reference.

The system controller 110 may be configured to determine the location ofthe mobile device 190 and/or the occupant 192. The system controller 110may be configured to control (e.g., automatically control) the loadcontrol devices (e.g., the lighting control devices of the lightingfixtures 172, 174, 176, 178, the plug-in load control device 140, themotorized window treatments 150, and/or the temperature control device160) in response to determining the location of the mobile device 190and/or the occupant 192.

One or more of the control devices of the load control system 100 maytransmit beacon signals, for example, RF beacon signals transmittedusing a short-range and/or low-power RF technology, such as BLUETOOTH®technology. The load control system 100 may also comprise at least onebeacon transmitting device 194 for transmitting the beacon signals. Themobile device 190 may be configured to receive a beacon signal whenlocated near a control device that is presently transmitting the beaconsignal. A beacon signal may comprise a unique identifier identifying thelocation of the load control device that transmitted the beacon signal.Since the beacon signal may be transmitted using a short-range and/orlow-power technology, the unique identifier may indicate the approximatelocation of the mobile device 190. The mobile device 190 may beconfigured to transmit the unique identifier to the system controller110, which may be configured to determine the location of the mobiledevice 190 using the unique identifier (e.g., using data stored inmemory or retrieved via the Internet). An example of a load controlsystem for controlling one or more electrical loads in response to theposition of a mobile device and/or occupant inside of a building isdescribed in greater detail in commonly-assigned U.S. Patent ApplicationPublication No. 2016/0056629, published Feb. 25, 2016, entitled LOADCONTROL SYSTEM RESPONSIVE TO LOCATION OF AN OCCUPANT AND MOBILE DEVICES,the entire disclosure of which is hereby incorporated by reference.

The visible light sensor 180 may comprise a camera directed into theroom 102 and may be configured to record images (e.g., still imagesand/or videos) of the room 102. For example, the visible light sensor180 may be mounted to a ceiling of the room 102, and/or may be mountedto a wall of the room (as shown in FIG. 1). The visible light sensor 180may comprise a fish-eye lens. If the visible light sensor 180 is mountedto the ceiling, the images recorded by the camera may be top down viewsof the room 102.

FIGS. 2A-2G show simplified example images of a room 200 that may berecorded by the camera of the visible light sensor. As shown in FIG. 2A,the room 200 may comprise room features. Room features may include walls210 having a doorway 212 and windows 214. The room 200 may include adesk 220 on which a computer monitor 222 and a keyboard 224 may belocated. The room 200 may also include a chair 226 on which an occupantof the room 200 may typically be positioned to use the computer monitor222 and the keypad 224. The example images of the room 200 shown inFIGS. 2A-2G are provided for informative purposes and may not beidentical to actual images captured by the visible light sensor 180.Since the visible light sensor 180 may have a fish-eye lens, the actualimages captured by the camera may warped images and may not be actualtwo-dimensional images as shown in FIGS. 2A-2G. In addition, the exampleimage of the room 200 shown in FIGS. 2A-2G show the walls 210 havingthickness and actual images captured by the visible light sensor 180 mayshow the interior surfaces of the room 102.

Referring again to FIG. 1, the visible light sensor 180 may beconfigured to process images recorded by the camera and transmit one ormore messages (e.g., digital messages) to the load control devices inresponse to the processed images. The visible light sensor 180 may beconfigured to detect one or more environmental characteristics of aspace (e.g., the room 102 and/or the room 200) from the images. Forexample, the control circuit of the visible light sensor 180 may beconfigured to evaluate an image and determine one or more environmentalcharacteristics within a room (e.g., room 102) depicted in the image.

Environmental characteristics may include one or more details of theimage, such as a movement, lighting intensity (e.g., lighting intensityfrom sunlight 196 and/or artificial light), color temperature, occupancyand/or vacancy condition, etc., depicted within the image. Lightingintensity may include a percentage of the light output by a lightingcontrol device. As described herein, the lighting intensity may includea lighting intensity from sunlight 196, artificial light, a percentageof the light output by a lighting control device, reflected light,luminance and/or illuminance. Luminance may include the amount of lightreflected from one or more surfaces and/or may indicate the luminouspower that may be perceived by the visible light sensor. Illuminance mayinclude the amount of light falling onto and/or spreading over one ormore surface areas. Luminance may be a measurable quantity. The visiblelight sensor 180 may determine an illuminance based (e.g., using acorrection factor) on a measured luminance. Luminance and illuminancemay correlate to the lighting intensity of a lighting fixture. Forexample, adjusting the lighting intensity of a lighting fixture mayaffect the quantity (e.g., measurable quantity) of the illuminance(e.g., the amount of light falling onto and/or spreading over one ormore surface areas). As the quantity of the illuminance changes, theluminance may change.

The visible light sensor 180 may be configured to determineenvironmental characteristics within the room using one or morealgorithms or image analysis techniques. For example, the visible lightsensor 180 may be configured to determine environmental characteristicswithin the room using background subtraction and/or backgroundmaintenance. The visible light sensor 180 may use background subtractionto detect objects that change within an image. For example, backgroundsubtraction may be used for detecting movement within an image and/orfor detecting an occupancy/vacancy condition within the image.Background maintenance may be used to perform background subtraction.Example algorithms that may be used to perform background maintenancemay include adjacent frame difference algorithms, mean and thresholdalgorithms, mean and covariance algorithms, mixture of Gaussianalgorithms, normalized block correlation algorithms, as well as others.The visible light sensor 180 may also, or alternatively, provide theimages to the system controller 110 or another computing device forperforming imaging analysis to determine environmental characteristicsand/or to control electrical loads/load control devices as describedherein.

The visible light sensor 180 may comprise a communication circuit fortransmitting and receiving the RF and/or wired signals. For example, thevisible light sensor 180 may comprise a communication circuit fortransmitting and receiving the RF signals 108 and/or the RF signals 109.The visible light sensor 180 may be configured to process one or moreimages recorded by the camera and transmit a digital message to the loadcontrol devices and/or to the system controller 110. The digitalmessages may include control instructions for controlling an electricalload at a corresponding load control device. The digital messages mayalso, or alternatively, include indications of environmentalcharacteristics identified in the images, from which controlinstructions may be generated for controlling an electrical load at aload control device. The visible light sensor 180 may transmit thedigital message to the load control devices and/or system controller ona periodic basis and/or based on another triggering event. The visiblelight sensor 180 may transmit the digital message to the load controldevices in response to a characteristic of the one or more images (e.g.,in response to one or more environmental characteristics determined fromthe images). For example, the visible light sensor 180 may be configuredto detect a movement, lighting intensity (e.g., lighting intensity fromsunlight 196 and/or artificial light), color temperature, and/oroccupancy/vacancy condition in the room 102 using the camera. Thevisible light sensor 180 may transmit a digital message to the loadcontrol devices and/or the system controller 110 via the RF signals 108(e.g., using the proprietary protocol) in response to detecting themovement, lighting intensity (e.g., lighting intensity from sunlight 196and/or artificial light), color temperature, and/or occupancy/vacancyconditions.

The visible light sensor 180 may operate to configure and/or control theload control system 100. The visible light sensor 180 may generateimages and identify and/or define objects in the images for enablingcontrol of the devices in the load control system. The visible lightsensor 180 may identify movements, light intensities, colortemperatures, occupancy/vacancy conditions from the objects in theimages. The load control system 100 may be configured according to thedefined objects, movements, light intensities, color temperatures,occupancy/vacancy conditions and rules that are defined thereon.

The visible light sensor 180 may be configured to operate in one or moresensor modes (e.g., an occupancy/vacancy sensor mode, a daylightingsensor mode, a color sensor mode, a daylight glare sensor mode, anoccupant count sensor mode, etc.). The visible light sensor 180 mayexecute different algorithms to process the images in each of the sensormodes to determine data to transmit to the load control devices. Thevisible light sensor 180 may transmit digital messages via the RFsignals 108 (e.g., using the proprietary protocol) in response to theimages. The visible light sensor 180 may send the digital messages(e.g., control instructions) directly to the load control devices and/orto the system controller 110 which may then communicate the messages tothe load control devices. The visible light sensor 180 may comprise afirst communication circuit for transmitting and/or receiving the RFsignals 108 using a proprietary protocol.

A user 192 may configure the visible light sensor 180 to perform actionswithin the room 102 according to the daylight glare sensor mode,daylighting sensor mode, color sensor mode, occupancy/vacancy sensormode, and/or occupant count sensor mode. The user 192 may configure thevisible light sensor 180 to perform actions according to the daylightglare sensor mode, daylighting sensor mode, color sensor mode,occupancy/vacancy sensor mode, and/or occupant count sensor mode withinone or more regions of interest within the room 102. For example, theuser 192 may configure the visible light sensor 180 to set the totallighting intensity (e.g., artificial light and/or sunlight 196) to apreferred total illuminance a task area, based on the user 192 enteringthe room 102, exiting the room 102, and/or residing within the room 102.The user 192 may additionally, or alternatively, configure the visiblelight sensor 180 to set the color temperature to a preferred colortemperature, based on the user 192 entering the room 102, exiting theroom 102, and/or residing within the room 102. The visible light sensor180 may apply one or more digital masks within room 102 when indifferent modes. Each sensor mode may have different masks that may beapplied when the visible light sensor 180 operates in the correspondingmode.

The visible light sensor 180 may be configured to perform a plurality ofsensor events to detect various environmental characteristics of thespace. For example, to perform a sensor event, the visible light sensor180 may be configured to operate in one or more sensor modes. Eachsensor mode, when executed, may detect one or more sensor events. Asensor event may be detected using an algorithm that identifies one ormore environmental characteristics in an image. For example, in anoccupancy/vacancy sensor mode, a sensor event may include entry of auser into a doorway of a room, movement detected within a predefinedarea of a room, or another occupancy/vacancy sensor event that may bedetected from the environmental characteristics of the space. Inaddition, the visible light sensor 180 may configured to obtain frommemory certain pre-configured control parameters (e.g., sensitivity,baseline values, threshold values, limit values, etc.) that may be usedby the algorithm to detect the environmental characteristic during thesensor event.

The visible light sensor 180 may be configured to focus on one or moreregions of interest in the image recorded by the camera when processingthe image to detect the environmental characteristic during the sensorevent. For example, certain areas of the image recorded by the cameramay be masked (e.g., digitally masked), such that the visible lightsensor 180 may not process the portions of the image in the maskedareas. When certain environmental characteristics of a sensor event areidentified in the unmasked portion of the image, a control strategy maybe triggered. The control strategy may be an algorithm for performingcontrol (e.g., generating control instruction) of one or more loadcontrol devices based on the detected environmental characteristics.

A region of interest may be a region within the room 102 that may berelevant to the environmental characteristics within the room 102. Forexample, a region of interest may be the door 105 (e.g., or other roomfeatures), a user task area (e.g., the desk 106, monitor 166, and/orkeyboard 168), a user's path from the door 105 to the user task area,etc. The visible light sensor 180 may be configured to determine one ormore environmental characteristics present at the region of interest.For example, the visible light sensor 180 may be configured to determinelighting intensity at a user task area. The visible light sensor 180 maydetermine the lighting intensity at the user task area, for example, todetermine if the lighting intensity present at the task area is apreferred lighting intensity. As another example, the visible lightsensor 180 may be configured to determine an occupancy/vacancy conditionat the path from the door 105 to the user task area. The visible lightsensor may be configured to determine the occupancy/vacancy condition toadjust control devices (e.g., lighting fixtures 172, 174, 176, 178)based on whether the user 192 is entering the room 102, exiting the room102, or residing within the room 102.

The visible light sensor 180 may be configured to provide a mask (e.g.,digital mask) within the room 102. The visible light sensor 180 may beconfigured to apply a mask (e.g., a predetermined digital mask that maybe stored in memory) to focus on a specific region of interest, andprocess the portion of the image in the region of interest. In addition,the visible light sensor 180 may be configured to focus on multipleregions of interest in the image at the same time (e.g., as shown inFIGS. 2B-2G). For example, the visible light sensor 180 may provide amask over the door 105 in the room 102. With the door 105 being masked,the visible light sensor 180 may disregard the door 105 and/or movementlocated at the door 105. If a portion of the room 102 is masked, thevisible light sensor 180 may focus on one or more regions of interest.For example, if a door 105 is masked, the visible light sensor 180 mayfocus on a user task area, such as the desk 106, monitor 166, andkeyboard 168 (e.g., a user area that is not masked). Specific mask(s)may be defined for each sensor event.

Image processing (e.g., digital image processing) may be performed todigitally mask one or more portions of the room 102. For example, imageprocessing may digitally mask one or more portions of the room 102 byselecting a set of pixels within the image for which processing by thevisible light sensor 180 may, or may not, take place. The visible lightsensor 180 may record an image of the room 102 and digitally mask aportion (e.g., the door 105) of the room 102 by disregarding one or moreof the pixels of the image that represent the portion of the room 102 tobe digitally masked.

A mask may be used to disregard portions of a space (e.g., the room 102)that may be less relevant, or less relevant for a period of time, forcontrolling the load control system 100. For example, a mask may be usedto disregard portions of the room (e.g., the door 105 within room 102and/or internal windows in the room 102) to block activity (e.g.,walking and/or lighting in a hallway adjacent to room 102) occurringoutside of the room 102. The visible light sensor 180 may mask the door105 within room 102 and/or internal windows in the room 102 bydisregarding pixels of the room 102 that depict objects and/or activitynear the door 105.

The mask may be used to disregard one or more objects within the room102. For example, a mask may be used to disregard portions of the roomother than the door 105 within room 102. The door 105 may be monitoredto identify an occupancy condition. For example, the visible lightsensor 180 may be configured to monitor an occupancy condition byidentifying users walking in and/or out of the door 105 of the room 102.The visible light sensor 180 may control the load control system 100according to the occupancy condition. The visible light sensor 180 maybe unresponsive to a movement and/or user in the masked areas whendetermining an occupancy/vacancy condition in the room 102. The visiblelight sensor 180 may be configured to exclude detection of motion withinone or more portions of the room 102 if a movement and/or user withinthe portion of the room 102 is irrelevant to the load control system100.

The visible light sensor 180 may be configured to dynamically changebetween the sensor modes, apply digital masks to the images, and controlparameters depending upon the present sensor event. For example, thevisible light sensor 180 may be configured to sequentially and/orperiodically step through the different sensor modes during operation(e.g., the occupancy/vacancy sensor mode, the daylighting sensor mode,the color sensor mode, the daylight glare sensor mode, the occupantcount mode, etc.). Each sensor event may be characterized by a sensormode (e.g., specifying an algorithm to use), one or more controlparameters, and/or one or more digital masks. The sensor event may bedetected during a sensor mode when environmental characteristics areidentified in an unmasked area of the image.

The visible light sensor 180 may be configured to cycle through thedifferent sensor modes during operation after expiration of a period oftime (e.g., via a round robin technique, such as giving each sensor modea predefined amount of time in sequence before returning to the firstsensor mode of the sequence). The visible light sensor 180 may beconfigured to change its sensor mode depending upon the presentenvironmental characteristic being identified from the images, orchanges in an environmental characteristic being identified from theimages. For example, the visible light sensor 180 may be configured tochange its sensor mode depending on user movements, light levels insideof the room 102, daylight levels outside of the room 102, colortemperature, daylight glare, etc. The visible light sensor 180 may beconfigured to change its sensor mode depending on an occupancy/vacancycondition. For example, the visible light sensor 180 may operate in theoccupancy/vacancy sensor mode if the room 102 is vacant and no othersensor modes may be used during a vacancy condition. The sensor modesmay be organized according to a prioritized set, for example, that isdefined during configuration of the visible light sensor 180.

The visible light sensor 180 may apply different masks to portions ofthe room 102. The different masks may operate during different modes ofoperation or relate to different objects for performing control. Thedifferent masks may operate during the same sensor mode or relate to thesame objects for performing control. For example, the visible lightsensor 180 may be configured to identify an environmental characteristicof a first region of interest by applying a first mask to objects otherthan the first region of interest. The first mask may allow the visiblelight sensor to, for example, focus on a first region of interest anddetect a color intensity within the first region of interest. Thevisible light sensor 180 may be configured to disregard environmentalcharacteristics of a second region of interest by applying the secondmask to disregard objects other than the second region of interest. Forexample, the second mask may relate to disregarding a movement in anarea other than the second region of interest. The visible light sensor180 may be configured to apply the first mask to focus on the firstregion of interest of the image in order to detect at least one of amovement, a color temperature, an occupancy/vacancy condition, etc. inthe first region of interest. The visible light sensor 180 may beconfigured to apply the second mask to disregard the second region ofinterest of the image in order to disregard a lighting intensity, acolor temperature, an occupancy/vacancy condition, etc. in the secondregion of interest.

A first region of interest may be a region in which a user performs atask, such as a user's task area. For example, the first region ofinterest may include a user's desk 106, monitor 166, and/or keyboard168. The second region of interest may be a region in which the userperforms a task, or the second region of interest may be a region ofcontrol unrelated to the user performing a task. For example, a secondregion of interest may be doorway 105, window 104, and/or anotherlocation within a space in which the control circuit is to disregard amovement, lighting intensity (e.g., lighting intensity from sunlight 196and/or artificial light), color temperature, and/or occupancy/vacancycondition. The visible light sensor 180 may be configured to apply afirst mask to the first region of interest (e.g., the user's desk 106,monitor 166, and/or keyboard 168) and/or the visible light sensor 180may be configured to apply a second mask to the second region ofinterest (e.g., doorway 108, window 104).

Objects may be identified and/or defined with each of the one or moreregions of interest using the images of the regions. Movements, lightintensities, color temperatures, and/or occupancy/vacancy conditions maybe determined from the objects identified and/or defined with theregions of interest. For example, a lighting intensity may be definedwithin the region of interest relating to the user's desk 106 that maybe different than the lighting intensity that may be defined within theregion of interest relating to the pathway from the door 105 to theuser's desk. Portions of a region of interest may be defined as aportion within one or more other regions of interest. For example,keyboard 168 may be defined as a region of interest and/or a desk 106may be defined as a separate region of interest even though the keyboard168 is located within the region of interest defined by the desk 106.Although a region of interest may be located within another region ofinterest, each of the regions of interest may be individually definedand/or controlled via the visible light sensor 180 (e.g., via controlinstructions and/or indications). For example, although the keyboard 168is located within the region of interest defined by the desk 106, theload control devices providing lighting to the keyboard 168 may bedefined and/or controlled independently from the load control devicesproviding lighting to the desk 106.

The visible light sensor 180 may be configured to focus on multipleregions of interest when detecting an occupancy/vacancy condition withinroom 102. The visible light sensor 180 may be configured to apply one ormore masks to one or more portions of the room 102. For example, thevisible light sensor 180 may be configured to apply a first mask to afirst portion of a room 102 (e.g., the window 104) to focus on motionwithin a first region of interest (e.g., the user's desk 106), a secondmask to a second portion of the room 102 (e.g., the desk 106) to focuson motion within a second region of interest (e.g., a path from the door105 to the user's desk 106), and a third mask to a third portion of theroom 102 to disregard motion within a third region of interest (e.g., adoorway, such as door 105). The visible light sensor 180 may beconfigured to control one or more control devices based on theoccupancy/vacancy condition. For example, the visible light sensor 180may be configured to determine a user 192 is occupying a region ofinterest and the visible light sensor 180 may be configured to providelighting (e.g., from lighting fixtures 172, 174, 176, 178) at the regionof interest.

The visible light sensor 180 may be configured to operate in theoccupancy/vacancy sensor mode to determine an occupancy and/or vacancycondition in the space in response to detection of movement within oneor more regions of interest. While in the occupancy/vacancy sensor mode,the visible light sensor 180 may be configured to use an occupancyand/or vacancy detection algorithm to determine that the space isoccupied in response to the amount of movement and/or the velocity ofmovement exceeding an occupancy threshold.

The visible light sensor 180 may compare a recorded image of the room102 with one or more other recorded images (e.g., previously recordedimages and/or subsequently recorded images) of the room 102 to determinewhether there are differences. For example, the visible light sensor 180may compare a recorded image of the room 102 with one or more otherrecorded images of the room 102 to determine if a user 192 has enteredthe room and/or if the user 192 has exited the room 102. For example, ifa user 192 appears in an image of the room 102 the visible light sensor180 may determine that an occupancy condition has occurred. If the userdisappears from an image of the room 102 the visible light sensor 180may determine that a vacancy condition has occurred.

During a sensor event for detecting occupancy and/or vacancy, thevisible light sensor 180 may be configured to apply a predetermined maskto focus on one or more regions of interest in one or more imagesrecorded by the camera and determine occupancy or vacancy of the spacebased on detecting or not detecting motion in the regions of interest.The visible light sensor 180 may be responsive to movement in theregions of interest and not be responsive to movement in the masked-outareas.

As shown in FIG. 2B, the visible light sensor 180 may be configured toapply a mask 230 to an image of the room 200 to exclude detection ofmotion in the doorway 212 and/or the windows 214, and may focus on aregion of interest 232 that include the interior space of the room 200.The visible light sensor 180 may be configured to apply a first mask tofocus on a first region of interest, apply a second mask to focus on asecond region of interest, and determine occupancy or vacancy based onmovement detected in either of the regions of interest. In addition, thevisible light sensor 180 may be configured to focus on multiple regionsof interest in the image at the same time by applying different masks tothe image(s).

Also, or alternatively, the visible light sensor 180 may identify a userpath when the visible light sensor 180 is in the occupancy/vacancysensor mode. The user path may be a predefined location and/or directionwithin the room 200 that the user 192 may be located and/or that theuser 192 may move within the room 200. For example, the user path may bea position and/or direction that a user 200 may take, or has beenidentified as taking, when walking from the doorway 212 towards thechair 250. The user path may be illuminated when occupancy/vacancy isidentified in the room 200.

The visible light sensor 180 may be configured to adjust certain controlparameters (e.g., sensitivity) to be used by the occupancy and/orvacancy algorithm depending upon the present sensor event. The occupancythreshold may be dependent upon the sensitivity. For example, thevisible light sensor 180 may be configured to be more sensitive or lesssensitive to movements in a first region of interest than in a secondregion of interest.

As shown in FIG. 2C, the visible light sensor 180 may be configured toincrease the sensitivity and apply a mask 240 to focus on a region ofinterest 242 around the keyboard 224 to be more sensitive to movementsaround the keyboard. In other words, by using masks that focus on“smaller” vs “larger” (e.g., the keyboard vs. the desk surface on whichthe keyboard may sit), the visible light sensor 180 may be configured toincrease and/or decrease the sensitivity of detected or not detectedmovements. In addition, through the use of masks, visible light sensor180 may be configured to not simply detect movement in the space, butdetect where that movement occurred.

The visible light sensor 180 may be configured to determine an occupancyand/or vacancy condition in the space in response to an occupant movinginto or out of a bounded area. For example, as shown in FIG. 2D, thevisible light sensor 180 may be configured to determine an occupancycondition in the room 200 in response to the occupant crossing aboundary of a bounded area 250 surrounding the chair 226 to enter thebounded area. After the occupant crosses the boundary, the visible lightsensor 180 may assume that the space is occupied (e.g., independent ofother sensor events of occupancy and/or vacancy) until the occupantleaves the bounded area 250. The visible light sensor 180 may not beconfigured to determine an occupancy condition in the room 200 until theoccupant crosses the boundary of the bounded area 250 to exit thebounded area. After the occupant leaves the bounded area, the visiblelight sensor 180 may be configured to detect a vacancy condition, forexample, in response to determining that there is no movement in theregion of interest 232 as shown in FIG. 2B. Thus, the visible lightsensor 180 may maintain the occupancy condition even if the movement ofthe occupant comprises fine movements (e.g., if the occupant is sittingstill or reading in the chair 226) or no movements (e.g., if theoccupant is sleeping in a bed).

The bounded area may surround other structures in different types ofrooms (e.g., other than the room 200 shown in FIG. 2D). For example, ifthe bounded area surrounds a hospital bed in a room, the systemcontroller 110 may be configured to transmit an alert to the hospitalstaff in response to the detection of movement out of the region ofinterest (e.g., indicating that the patient got up out of the bed). Inaddition, the visible light sensor 180 may be configured count thenumber of occupants entering and exiting a bounded area.

Referring again to FIG. 1, the visible light sensor 180 may transmitdigital messages to the system controller 110 via the RF signals 108(e.g., using the proprietary protocol) in response to detecting theoccupancy or vacancy conditions. The system controller 110 may beconfigured to turn the lighting loads (e.g., lighting loads in lightingfixtures 172, 174, 176, 178 and/or the lighting load in the floor lamp142) on and off in response to receiving an occupied command and avacant command, respectively. Alternatively, the visible light sensor180 may transmit digital messages (e.g., including control instructions)directly to the lighting control devices for the lighting loads (e.g.,lighting control devices for the lighting fixtures 172, 174, 176, 178,plug-in load control device 140, etc.). The visible light sensor 180 mayoperate as a vacancy sensor, such that the lighting loads are onlyturned off in response to detecting a vacancy condition (e.g., and notturned on in response to detecting an occupancy condition). Examples ofRF load control systems having occupancy and vacancy sensors aredescribed in greater detail in commonly-assigned U.S. Pat. No.8,009,042, issued Aug. 30, 2011, entitled RADIO-FREQUENCY LIGHTINGCONTROL SYSTEM WITH OCCUPANCY SENSING; U.S. Pat. No. 8,199,010, issuedJun. 12, 2012, entitled METHOD AND APPARATUS FOR CONFIGURING A WIRELESSSENSOR; and U.S. Pat. No. 8,228,184, issued Jul. 24, 2012, entitledBATTERY-POWERED OCCUPANCY SENSOR, the entire disclosures of which arehereby incorporated by reference.

The visible light sensor 180 may be configured to adjust (e.g., viacontrol instructions) one or more light sources (e.g., lighting fixtures172, 174, 176, 178) during the occupancy/vacancy mode based on anoccupancy and an activity being performed within the room 102. Forexample, the visible light sensor 180 may determine that user 192 isoccupying the room 102 and that the user 192 is typing or writing on atask area (e.g., desk 106). The visible light sensor 180 may adjust thelighting fixtures 172, 174, 176, 178 to provide a desired amount oflighting to the desk 106 according to the identified activity. Forexample, the visible light sensor 180 may be configured to provide morelighting to the desk 106 when the user 192 is writing or typing at thedesk 106 than when the user 192 is not occupying the room 102 or whenthe user 192 is performing another activity at the desk 106.

The visible light sensor 180 may also be configured to operate in thedaylighting sensor mode to measure a lighting level (e.g., illuminanceor luminance due to daylight and/or artificial light) at a location ofthe space. For example, the visible light sensor 180 may apply a digitalmask to focus on a specific location in the space (e.g., on a task area,such as a surface or a table 106 as shown in FIG. 1) and may use adaylighting algorithm to measure the lighting level at the location.Since the camera of the visible light sensor 180 is directed towards thesurface, the visible light sensor may be configured to measure theluminance (e.g., reflected light level) at the surface. The visiblelight sensor 180 may be configured to calculate lighting level levelsusing image data. Image data may include data associated with thelighting level and/or color of pixels in the image. The visible lightsensor 180 may calculate the illuminance (e.g., lighting level shiningon the surface) from the measured luminance using a conversion factor.The conversion factor may be determined during a calibration procedureof the visible light sensor 180. For example, the illuminance at thetask surface may be measured by a light meter and may be transmitted tothe visible light sensor 180. The visible light sensor 180 may beconfigured to measure the luminance at the surface and may be configuredto determine the conversion factor as a relationship between theilluminance measured by the light meter and the luminance measured bythe visible light sensor.

As shown in FIG. 2E, the visible light sensor 180 may be configured toapply a mask 260 to focus on a region of interest 262 that includes thesurface of the desk 220. The visible light sensor 180 may be configuredto integrate light intensities values of the pixels of the image acrossthe region of interest 262 to determine a measured lighting intensity orcolor at the surface of the desk.

The visible light sensor 180 may disregard the objects within the roomif it is determined that the objects are inconsistent with other objectswithin the room 102. For example, the visible light sensor 180 maydisregard objects (e.g., one or more pieces of white paper, books,monitors, keyboards, computers, or other objects) located on the desk220. The objects located on a desk 220 may be presented with brightnessor color that is different from the brightness or color being reflectedoff of the desk 220 on which the objects are located. The visible lightsensor 180 may determine that the objects on the desk 220 are not a partof the desk. Thus, when identifying attributes of the desk 220 (e.g.,the size, shape, location, etc.), the visible light sensor 180 may maskthe objects located on the desk 220. The objects on the desk 220 may bemasked to control the load control system 100 according to the intensityor color reflected off of the uncovered portions of the desk 220.

Referring again to FIG. 1, the visible light sensor 180 may transmitdigital messages (e.g., including the measured lighting intensity) tothe system controller 110 via the RF signals 108 for controlling theintensities of the lighting loads (e.g., lighting loads in lightingfixtures 172, 174, 176, 178 and/or the lighting load in the floor lamp142) in the load control environment 100 in response to the measuredlighting intensity. The visible light sensor 180 may be configured tofocus on multiple regions of interest in the image recorded by thecamera and measure the lighting intensity in each of the differentregions of interest. Alternatively, the visible light sensor 180 maytransmit digital messages directly to the lighting control devices forthe lighting loads (e.g., lighting control devices for the lightingfixtures 172, 174, 176, 178, plug-in load control device 140, etc.). Thevisible light sensor 180 may be configured to adjust certain controlparameters (e.g., gain) based on the region of interest in which thelighting intensity is presently being measured. Examples of RF loadcontrol systems having daylight sensors are described in greater detailin commonly-assigned U.S. Pat. No. 8,410,706, issued Apr. 2, 2013,entitled METHOD OF CALIBRATING A DAYLIGHT SENSOR; and U.S. Pat. No.8,451,116, issued May 28, 2013, entitled WIRELESS BATTERY-POWEREDDAYLIGHT SENSOR, the entire disclosures of which are hereby incorporatedby reference.

The visible light sensor 180 may determine whether the lightingintensity at the region of interest is different (e.g., higher, lower)than a preferred total lighting intensity at the region of interest. Forexample, the visible light sensor 180 may be configured to determine ifthe lighting intensity presented on a user's task area (e.g., a desk106, predefined distance around the user 192, monitor 166, etc.) is apreferred total lighting intensity. The preferred total lightingintensity may be provided as a default preferred total intensity. Thepreferred total lighting intensity may be provided by the user 192(e.g., via mobile device 190 used by the user).

The system controller 110 may be configured to determine a degradationin the light output of one or more of the lighting loads (e.g., lightingloads in lighting fixtures 172, 174, 176, 178 and/or the lighting loadin the floor lamp 142) in the space, and to control the intensities ofthe lighting loads to compensate for the degradation (e.g., lumenmaintenance). For example, the system controller 110 may be configuredto individually turn on each lighting load (e.g., when it is dark atnight) and measure the magnitude of the lighting intensity at a location(e.g., on the table 106 or the desk 220). For example, the systemcontroller 110 may be configured to turn on the lighting loads at nightand control the visible light sensor 180 to record an image of the room,to apply a mask to focus on a region of interest that the lighting loadsilluminate (e.g., the surface of table 106 or the desk 220), to measurethe lighting intensity in that region of interest, and to communicatethat value to the system controller 110. The system controller 110 maystore this value as a baseline value. At a time and/or date thereafter,the system controller 110 may repeat the measurement and compare themeasurement to the baseline value. If the system controller 110determines there to be a degradation, it may control one or more of thelighting loads to compensate for the degradation, alert maintenance,etc.

The visible light sensor 180 may also be configured to operate in thecolor sensor mode to detect a color (e.g., measure a color temperature)of the light emitted by one or more of the lighting loads in the space(e.g., to operate as a color sensor and/or a color temperature sensor).For example, as shown in FIG. 2F, the visible light sensor 180 may beconfigured to apply a mask 270 to focus on a region of interest 272(that includes a portion of the surface of the desk 220) and may use acolor sensing algorithm to determine a measured color and/or colortemperature in the room 200. For example, the visible light sensor 180may integrate color values of the pixels of the image across the regionof interest 272 to determine the measured color and/or color temperaturein the room 200. The region of interest 272 may include a portion of thedesk having a known color (e.g., white), or a color wheel having colorsfor which RGB values may be identified.

Referring again to FIG. 1, the visible light sensor 180 may transmitdigital messages (e.g., including the measured color temperature) to thesystem controller 110 via the RF signals 108 for controlling the color(e.g., the color temperatures) of the lighting loads (e.g., lightingloads in lighting fixtures 172, 174, 176, 178 and/or the light in thefloor lamp 142) in response to the measured lighting intensity (e.g.,color tuning of the light in the space). Alternatively, the visiblelight sensor 180 may transmit digital messages directly to the lightingloads. An example of a load control system for controlling the colortemperatures of one or more lighting loads is described in greaterdetail in commonly-assigned U.S. Patent Application Publication No.2014/0312777, published Oct. 23, 2014, entitled SYSTEMS AND METHODS FORCONTROLLING COLOR TEMPERATURE, the entire disclosure of which is herebyincorporated by reference.

The visible light sensor 180 may be configured to operate in a daylightglare sensor mode. For example, the visible light sensor 180 may beconfigured execute a glare detection algorithm to determine a depth ofdirect sunlight penetration into the space from the image recorded bythe camera. As shown in FIG. 2G, the visible light sensor 180 may beconfigured to apply a mask 280 to focus on a region of interest 282 onthe floor of the room 200 near the windows 214 to detect the depth ofdirect sunlight penetration into the room.

Referring again to FIG. 1, the visible light sensor 180 may mask one ormore objects in the room 102, besides the task area, when in thedaylight glare sensor mode. For example, the masked images from thevisible light sensor 180 may show the desk 106, keyboard 168, andmonitor 166 when the visible light sensor 180 is in the daylight glaresensor mode. Also, or alternatively, the visible light sensor 180 maymask one or more objects in the room 102, outside of a predefined areaaround the task area, when in the daylight glare sensor mode. Thevisible light sensor 180 may retain the task area and/or the predefinedarea around the task area to determine whether sunlight penetration hasreached the task area and/or the predefined area around the task area.

Based on a detection and/or measurement of the depth of direct sunlightpenetration into the room, the visible light sensor 180 may transmitdigital messages to the system controller 110 via the RF signals 108 tolimit the depth of direct sunlight penetration into the space, forexample, to prevent direct sunlight from shining on a surface (e.g., thetable 106 or the desk 220). The system controller 110 may be configuredto lower the window treatment fabric 152 of each of the motorized windowtreatments 150 to prevent the depth of direct sunlight penetration fromexceeded a maximum sunlight penetration depth. Alternatively, thevisible light sensor 180 may be configured to directly control thewindow treatments 150 to lower of the window treatment fabric 152.Examples of methods for limiting the sunlight penetration depth in aspace are described in greater detail in commonly-assigned U.S. Pat. No.8,288,981, issued Oct. 16, 2012, entitled METHOD OF AUTOMATICALLYCONTROLLING A MOTORIZED WINDOW TREATMENT WHILE MINIMIZING OCCUPANTDISTRACTIONS, the entire disclosure of which is hereby incorporated byreference.

During daylight glare sensor mode, the visible light sensor 180 may beconfigured to control the covering material 152 of the motorized windowtreatments 150 to prevent daylight glare from reaching a region ofinterest, such as the user's task area. The visible light sensor 180 maydetermine whether an intensity of light presented at a region ofinterest is a preferred intensity of light. For example, a user 192 maydesire that sunlight 196 be prevented from reaching a region of interestthat is a task area (e.g., a desk 106, predefined distance around theuser 192, monitor 166, etc.). The visible light sensor 180 may beconfigured to determine if an undesired amount of sunlight is beingpresented to a region of interest (e.g., task area) by comparing thelighting intensity and/or color temperature presented near a window 104with the lighting intensity and/or color temperature presented onanother portion of the room 102 that is away from a window 104. Forexample, it may be determined that the light presented near a window 104is a result of sunlight 196 (e.g., based on the images, the colortemperature of the light in the images, the control settings for thelighting fixtures near the window, etc.) and it may be determined thatthe light presented away from a window 104 is the result of sources oflight other than sunlight 196 (e.g., based on the images, the colortemperature of the light in the images, the control settings for thelighting fixtures away from the window, etc.).

The visible light sensor 180 may be configured to focus on daylightentering the space through, for example, one or both of the windows 104(e.g., to operate as a window sensor). The system controller 110 may beconfigured to control the lighting loads (e.g., lighting loads inlighting fixtures 172, 174, 176, 178 and/or the light in the floor lamp142) in response to the magnitude of the daylight entering the space.The system controller 110 may be configured to override automaticcontrol of the motorized window treatments 150, for example, in responseto determining that it is a cloudy day or an extremely sunny day.Alternatively, the visible light sensor 180 may be configured todirectly control the window treatments 150 to lower of the windowtreatment fabric 152. Examples of load control systems having windowsensors are described in greater detail in commonly-assigned U.S. PatentApplication Publication No. 2014/0156079, published Jun. 5, 2014,entitled METHOD OF CONTROLLING A MOTORIZED WINDOW TREATMENT, the entiredisclosure of which is hereby incorporated by reference.

The visible light sensor 180 may be configured to detect a glare source(e.g., sunlight reflecting off of a surface) outside or inside the spacein response to the image recorded by the camera. The system controller110 may be configured to lower the window treatment fabric 152 of eachof the motorized window treatments 150 to eliminate the glare source.Alternatively, the visible light sensor 180 may be configured todirectly control the window treatments 150 to lower of the windowtreatment fabric 152 to eliminate the glare source.

The visible light sensor 180 may also be configured to operate in theoccupant count mode and may execute an occupant count algorithm to countthe number of occupants a particular region of interest, and/or thenumber of occupants entering and/or exiting the region of interest. Theoccupant count algorithm may identify an environmental characteristicsand/or may that trigger a sensor event for executing a control strategy.For example, the system controller 110 may be configured to control theHVAC system 162 in response to the number of occupants in the space. Thesystem controller 110 may be configured to control one or more of theload control devices of the load control system 100 in response to thenumber of occupants in the space exceeding an occupancy numberthreshold. Alternatively, the visible light sensor 180 may be configuredto directly control the HVAC system 162 and other load control devices.

The operation of the load control system 100 may be programmed andconfigured using, for example, the mobile device 190 or other networkdevice (e.g., when the mobile device is a personal computing device).The mobile device 190 may execute a graphical user interface (GUI)configuration software for allowing a user to program how the loadcontrol system 100 will operate. For example, the configuration softwaremay run as a PC application or a web interface. The configurationsoftware and/or the system controller 110 (e.g., via instructions fromthe configuration software) may generate a load control database thatdefines the operation of the load control system 100. For example, theload control database may include information regarding the controlsettings of different load control devices of the load control system(e.g., the lighting fixtures 172, 174, 176, 178, the plug-in loadcontrol device 140, the motorized window treatments 150, and/or thethermostat 160). The load control database may comprise informationregarding associations between the load control devices and the inputdevices (e.g., the remote control device 170, the visible light sensor180, etc.). The load control database may comprise information regardinghow the load control devices respond to inputs received from the inputdevices. Examples of configuration procedures for load control systemsare described in greater detail in commonly-assigned U.S. Pat. No.7,391,297, issued Jun. 24, 2008, entitled HANDHELD PROGRAMMER FOR ALIGHTING CONTROL SYSTEM; U.S. Patent Application Publication No.2008/0092075, published Apr. 17, 2008, entitled METHOD OF BUILDING ADATABASE OF A LIGHTING CONTROL SYSTEM; and U.S. Patent ApplicationPublication No. 2014/0265568, published Sep. 18, 2014, entitledCOMMISSIONING LOAD CONTROL SYSTEMS, the entire disclosure of which ishereby incorporated by reference.

A user 192 may configure the visible light sensor 180 to perform actionswithin one or more regions of interest the room 102 according to thedaylight glare sensor mode, daylighting sensor mode, color sensor mode,occupancy/vacancy sensor mode, and/or occupancy count mode. For example,the user 192 may configure the visible light sensor 180 to set the totallighting intensity (e.g., artificial light and/or sunlight) to apreferred total illuminance on a task area, based on the user 192entering the room 102, exiting the room 102, and/or residing within theroom 102. The user 192 may additionally, or alternatively, configure thevisible light sensor 180 to set the color temperature to a preferredcolor temperature, based on the user 192 entering the room 102, exitingthe room 102, and/or residing within the room 102.

The user 192 may provide user preferences (e.g., total intensitypreferences, color temperature preferences, etc.) using one or moreinput devices, such as mobile device 190. For example, in the daylightglare sensor mode, the user 192 may input a preferred amount of sunlightthat may present on the user task area when the user 192 is entering theroom 102, exiting the room 102, and/or residing within the room 102. Inthe daylighting sensor mode, the user 192 may input a preferred lightingintensity that lighting fixtures 172, 174, 176, 178 may present on theuser task area (e.g., desk 106, monitor 166, a predefined area arounduser 192, etc.) when the user 192 is entering the room 102, exiting theroom 102, and/or residing within the room 102. In the color sensor mode,the user 192 may input a preferred amount of color temperature that maypresent at the user task area when the user 192 is entering the room102, exiting the room 102, and/or residing within the room 102.

An image may be provided to the visible light sensor 180, via the mobiledevice 190, so that the visible light sensor 180 may identify the user192 entering the room 102, exiting the room 102, and/or residing withinthe room 102. The image may be recorded via a camera feature of themobile device 190. The image may be provided by an external server thatstores images of one or more users 192. For example, a company'sdatabase may include identification photographs of employees of thecompany. The visible light sensor 180 may be configured to receive theidentification photographs for identification of a user 192. The visiblelight sensor 180 may also, or alternatively, be configured to record animage of the user 192 for identification of the user 192. For example,during configuration of the visible light sensor 180, the user 192 mayprovide user preferences to the visible light sensor 180. The visiblelight sensor 180 may record an image of the user 192 while the user 192is configuring the visible light sensor 180. For example, the visiblelight sensor 180 may record an image of the user 192 while the user 192is providing user preferences to the visible light sensor 180. Inrecording an image of the user 192 while the user 192 is configuring thevisible light sensor 180, the visible light sensor 180 may make anassociation between the user 192 and the user preferences being used forconfiguration of the visible light sensor 180. The visible light sensor180 may record a still image of the user 192. The visible light sensor180 may record the user 192 performing a movement, such as walkingwithin the room 102 and/or performing an action (e.g., typing on thekeyboard 166 and/or writing on the desk 106) on the task surface orwithin a predefined distance of the task surface.

The visible light sensor 180 may comprise a second communication circuitfor transmitting and receiving the RF signals 109 (e.g., directly withthe mobile device 190 using a standard protocol, such as Wi-Fi orBluetooth). During the configuration procedure of the load controlsystem 100, the visible light sensor 180 may be configured to record animage of the space and transmit the image to the mobile device 190(e.g., directly to the network device via the RF signals 109 using thestandard protocol). The mobile device 190 may display the image on thevisual display and a user 192 may configure the operation of the visiblelight sensor 180 to set one or more configuration parameters (e.g.,configuration data) of the visible light sensor. For example, fordifferent environmental characteristic to be sensed for performingcontrol by the visible light sensor 180 (e.g., occupant movements, lightlevel inside of the room, daylight level outside of the room), the user192 may indicate different regions of interest on the image by tracing(such as with a finger or stylus) masked areas on the image displayed onthe visual display. The visible light sensor 180 may be configured toestablish different masks and/or control parameters depending upon theenvironmental characteristic to be sensed (e.g., occupant movements,light level inside of the room, daylight level outside of the room,color temperature, etc.).

After configuration of the visible light sensor 180 is completed at themobile device 190, the mobile device 190 may transmit configuration datato the visible light sensor (e.g., directly to the visible light sensorvia the RF signals 109 using the standard protocol). The visible lightsensor 180 may store the configuration data in memory, such that thevisible light sensor may operate appropriately during normal operation.For example, for each sensor event the visible light sensor 180 is tomonitor, the mobile device 190 may transmit to the visible light sensor180 the sensor mode for the event, one or more masks defining regions ofinterest for the event, possibly an indication of the algorithm to beused to sense the environmental characteristic of the event, and one ormore control parameters for the event.

The configuration data may include a room identifier or other identifierthat is stored for the configuration of the space. The configurationdata for a given room identifier or other identifier of a space may beused as a template (e.g., a configuration template) for configuring thevisible light sensor and/or load control within a similar space. Aconfiguration template may be copied and applied to other spaces forperforming load control. The configuration template may include similarmasks, regions of interest, control strategies, etc.

The visible light sensor 180 may be configured to provide a predefinedlighting intensity at one or more regions of interest. The predefinedlighting intensity may be the same or different among the regions ofinterest. The visible light sensor 180 may identify the sunlight and/orthe artificial light that comprises the lighting intensity provided tothe one or more regions of interest within the room 102. The visiblelight sensor 180 may increase or decrease the lighting intensity, orchange the color temperature, of the lighting fixtures 172, 174, 176,178 on a gradient across the room (e.g., from the windows 104, the door105, a projector screen, a television, or other presentation region).

The visible light sensor 180 may be configured to identify one or moreof the regions of interest within the room 102 using objects locatedwithin the room 102. For example, the visible light sensor 180 may beconfigured to identify a task area (e.g., a desk 106) using the size,shape, and/or location of the desk 106. That is, if the visible lightsensor 180 identifies an object having a predefined shape (e.g.,rectangular or circular), size, and/or location within the room 102, thevisible light sensor 180 may be configured to determine that the objectis a desk. Predefined sizes and/or shapes may be stored in memory forcomparison against the size of the objects identified in the images. Thevisible light sensor 180 may be configured to determine other objectswithin the room 102, for example, the door 105 and/or window 104, usingthe size, location, and/or orientation of the object. The visible lightsensor 180 may determine that an object is a door 105 if the object isthe predefined size of a door, located at a wall of the room 102, and/orthe orientation of the object represents a predefined orientation of adoor.

The visible light sensor 180 may be configured to determine a lightingintensity (e.g., sunlight, artificial light) that may be presented toone or more regions of interest. For example, the visible light sensor180 may be configured to determine sunlight that may be presented to aregion of interest. The visible light sensor 180 may be configured todetermine the sunlight that may be presented to a region of interestduring configuration of the load control system 100 and/or during use ofthe load control system 100. For example, the visible light sensor 180may determine that an undesired amount of sunlight is being presented ata first region of interest, such as a user's task area, duringconfiguration and/or control of the load control system 100. The visiblelight sensor 180 may determine the identification and/or location of auser task area by identifying a predefined location around the user 192.For example, the visible light sensor 180 may determine theidentification and/or location of a user task area by identifying apredefined a location around the user 192 for a predefined period oftime each day.

The visible light sensor 180 may determine the identification and/orlocation of a user task area automatically (e.g., using the size,location, and/or shape of the user area). For example, the visible lightsensor 180 may define a desk 106 using predefined sizes, shapes, and/orcolors of desks. The visible light sensor 180 may identify a particularuser's task area using attributes of the task area, and/or the visiblelight sensor 180 may identify a particular user's task area usingancillary objects (e.g., photos, mugs) placed on the user's task area.The visible light sensor 180 may be configured to control (e.g., viacontrol instructions) one or more control devices using theidentification and/or location of the user's task area. For example, thevisible light sensor 180 may be configured to determine the location ofa user's task area and present (e.g., via control instructions) apreferred lighting intensity from lighting fixtures 172, 174, 176, 178on the user's task area. The visible light sensor 180 may determine thelocation of a user's task area and control the covering material 152 ofthe motorized window treatments 150 so that a preferred amount ofdaylight glare is presented upon user's task area.

The visible light sensor 180 may be configured to determine whether auser's task area (e.g., desk 106, monitor 166, a predefined area arounduser 192, etc.) has moved. For example, the visible light sensor 180 maybe configured to determine whether a desk 106 has moved from one side ofthe room 102 to another side of the room 102. The visible light sensor180 may configured to determine whether one or more other task areas(e.g., desks 106, monitors 166, etc.) have been added to the room 102.For example, the visible light sensor 180 may be configured to compare arecorded image of the room 102 with one or more other recorded images(e.g., previously recorded images and/or subsequently recorded images)of the room 102 to determine whether there are differences, such asmovements of task areas and/or additions of task areas, within the room102. During configuration, the visible light sensor 180 may identifymovement of a user, an occupant, furniture, a partition, and/or otherobjects within the room. The visible light sensor 180 may be configuredto control one or more control devices based on movement of one or moreof the task areas (e.g., based on movement of the desk 106, monitor 166,etc.). For example, if a user's task area is moved, the visible lightsensor 180 may identify such movement and present the preferred lightingintensity and/or allow the preferred daylight glare at the updatedlocation of the task area.

The fixtures within the different regions of interest may be identifiedby the location of the lighting levels within the image. Differentfixtures may be mapped to different portions of the images generatedfrom the visible light sensor 180. The dimming level in the differentregions of interest may be adjusted by a predefined amount or may beadjusted to different dimming levels based on the difference in lightinglevels identified between the different regions. For example, thevisible lighting sensor 180 may change the dimming level of the lightingfixtures 174, 178 by 25% when the portion of the room 102 that includesthe sunlight 196 is determined to be 25% brighter. Examples of RF loadcontrol systems having daylight sensors are described in greater detailin commonly-assigned U.S. Pat. No. 8,410,706, issued Apr. 2, 2013,entitled METHOD OF CALIBRATING A DAYLIGHT SENSOR; and U.S. Pat. No.8,451,116, issued May 28, 2013, entitled WIRELESS BATTERY-POWEREDDAYLIGHT SENSOR, the entire disclosures of which are hereby incorporatedby reference.

The visible light sensor 180 may be configured to determine one or moreregions of interest (e.g., zones) within the room 102 by determining thelocation of control devices positioned within the room 102. For example,the visible light sensor 180 may be configured to determine thatlighting control devices that provide a lighting load preferred forpresentations are positioned at a particular location within the room102. The visible light sensor 180 may be configured to identify thepresentation lighting loads for a presentation area and determine (e.g.,automatically determine) that the location of the presentation lightingloads is located within a presentation region of interest. The visiblelight sensor 180 may be configured to identify one or more motorizedwindow treatments 150 and determine (e.g., automatically determine) thatthe location of the motorized window treatments 150 may receiveadditional lighting intensity (e.g., sunlight).

The visible light sensor 180 may be configured to determine that thelighting control devices that have the same intended function aregrouped together in a region of interest (e.g., a zone). For example, agroup of functional lighting fixtures within a predefined distance ofthe windows may be grouped together in a daylighting zone. The visiblelight sensor 180 may be configured to determine that the lightingcontrol devices that illuminate a portion of the room 102 are groupedtogether in a region of interest (e.g., a zone). For example, a group oflighting fixtures that illuminate a task area (e.g., desks 106, monitors166, etc.) are grouped together in a zone. The visible light sensor 180may be configured to determine that the lighting control devices locatednear (e.g., within a predefined distance) an object or affecting thelighting level of an object are grouped together in a region of interest(e.g., a zone).

The visible light sensor 180 may be configured to operate in anoccupancy/vacancy sensor mode. In the occupancy/vacancy sensor mode, thevisible light sensor 180 may be configured to determine anoccupancy/vacancy condition (e.g., an environmental characteristic) ofone or more regions of interest. For example, the visible light sensor180 may determine an occupancy/vacancy condition of one or more regionsof interest based on detecting a presence or motion, or a lack ofpresence or motion, in the images captured in the regions of interest.The visible light sensor 180 may determine an occupancy/vacancycondition using one or more algorithms and/or image analysis techniques.For example, the visible light sensor 180 may determine anoccupancy/vacancy condition using background subtraction and/orbackground maintenance, as described herein. The visible light sensor180 may identify an occupancy/vacancy condition of one or more regionsof interest and control a load control device in response to theoccupancy/vacancy condition. For example, the visible light sensor 180may identify an occupancy condition in the room 102 when the user 192and/or the mobile device 190 enters the room 102 and may send controlinstructions to the lighting fixtures 172, 174, 176, 178, the plug-inload control device 140, the motorized window treatments 150, and/or thethermostat 160 for controlling an electrical load in response to theoccupancy condition. Examples of RF load control systems havingoccupancy/vacancy sensors are described in greater detail incommonly-assigned U.S. Pat. No. 8,009,042, issued Aug. 30, 2011,entitled RADIO-FREQUENCY LIGHTING CONTROL SYSTEM WITH OCCUPANCY SENSING;U.S. Pat. No. 8,199,010, issued Jun. 12, 2012, entitled METHOD ANDAPPARATUS FOR CONFIGURING A WIRELESS SENSOR; and U.S. Pat. No.8,228,184, issued Jul. 24, 2012, entitled BATTERY-POWERED OCCUPANCYSENSOR, the entire disclosures of which are hereby incorporated byreference.

The visible light sensor 180 may fail to identify an occupancy conditionbased on a movement and/or user presence detected within one or more ofthe regions of interest in which the visible light sensor 180 isdisregarding. For example, the visible light sensor 180 may apply a maskto a doorway and when identifying an occupancy condition the visiblelight sensor 180 may exclude movement and/or users at the doorway. If auser is not present within the room 102 and is standing outside of thedoorway, the visible light sensor 180 may fail to identify an occupancycondition within the room 102. The visible light sensor 180 maydetermine when the user moves outside of the masked area and into theregion of interest in which the visible light sensor 180 is configuredto determine an occupancy/vacancy condition. The masking of the doorway,windows, and/or other transparent spaces in the room 102 may prevent afalse identification of objects outside of the doorway, windows, and/orother transparent spaces.

The visible light sensor 180 may be configured to turn the lightingloads (e.g., lighting fixtures 172, 174, 176, 178) on and off inresponse to detecting an occupancy condition and a vacancy condition,respectively. The visible light sensor 180 may operate as a vacancysensor, such that the lighting loads are turned off in response todetecting a vacancy condition (e.g., and not turned on in response todetecting an occupancy condition).

The visible light sensor 180 may be configured to identify movementswithin a region of interest at a higher sensitivity than movements inone or more other regions of interest. For example, the visible lightsensor 180 may be configured to be more sensitive to movements in anarea around a user's task surface (e.g., keyboard) than in an area thatis less often used by the user 192. The visible light sensor 180 may beconfigured to increase the sensitivity to identify fingers moving on akeyboard 168 and/or a user 192 writing on desk 106, for example. Thevisible light sensor 180 may be unable to detect such minor motion inone or more other regions of interest within the room 102 to preventfalse indications of occupancy. If the visible light sensor 180identifies movement in the regions of interest in which the visiblelight sensor 180 is more sensitive, the visible light sensor 180 mayadjust the fixtures 172, 174, 176, 178 to provide increased lighting tothe sensitive areas (e.g., keyboard 168).

The visible light sensor 180 may be configured to determine anoccupancy/vacancy condition in the room 102 in response to a user movinginto a region of interest or out of a region of interest. For example, afirst region of interest may be a bed, an office, or a user task area. Asecond region of interest may be a path from the first region ofinterest to a door and/or another room (e.g., an office, a bathroom,etc.). The visible light sensor 180 may be configured to identify zonesof lights as being the lights in and/or between regions of interest. Thevisible light sensor 180 may define the lighting fixtures in the zonesin response to a user moving from one location to another (e.g., duringa configuration procedure or after identification of such a usermovement a predefined number of times). For example, the visible lightsensor 180 may be configured to identify the path of a user from a bedto the bathroom. The visible light sensor 180 may be configured todefine the lighting fixtures in the areas along the user's path in thesame zone for lighting control. The visible light sensor 180 mayincrease the light intensities provided by the lighting fixtures inresponse to a user moving in the direction of one of the regions ofinterest defined in the zone.

The visible light sensor 180 may be used with a passive infrared sensor(PIR) 182. The PIR sensor 182 may be an electronic device that measuresinfrared (IR) light radiating from one or more objects in the field ofview of the PIR sensor 182. The PIR sensor 182 may be used to identifymotion within the field of view of the PIR sensor 182. The PIR sensor182 may consume less power than the visible light sensor 180 and the PIRsensor 182 may be used to detect an occupancy/vacancy condition in theoccupancy/vacancy sensor mode. For example, the PIR sensor 182 may be alow-energy occupancy sensing circuit.

The PIR sensor 182 and the visible light sensor may operate incooperation, as each sensor may identify different types of information.For example, the PIR sensor 182 may operate to trigger the visible lightsensor 180, as the PIR sensor 182 may reduce the number of falseidentifications of occupancy. The PIR sensor 182 may operate to triggerthe visible light sensor 180 so that the visible light sensor 180 may bebegin recording images and/or controlling one or more control deviceswithin room 102. As the visible light sensor 180 may operate to detectoccupancy by the movement of objects within images, the movement ofobjects other than a user may trigger an occupancy condition at thevisible light sensor 180. The PIR sensor 182, however, may detectmovement of a user in a room using infrared signals. The infraredsignals may be used to trigger the visible light sensing circuit, whichmay more accurately track objects after occupancy has been determined.For example, an infrared signal may cause the lighting fixtures to turnoff when a user makes little or no movement (e.g., minor motion events)for a period of time. The visible light sensor may be able to identifythe presence of the user in the images, even though the user may makelittle or no movement (e.g., minor motion events) for a period of time.

The visible light sensor 180 may conserve power and/or storage used tostore images by enabling a visible light sensing circuit when movementis detected by the PIR sensor 182. When movement is detected by the PIRsensor 182, the visible light sensing circuit may be enabled forgenerating images of the space and detecting users and/or movement. Thevisible light sensing circuit may operate in place of, or in additionto, the PIR sensor 182 for identifying occupancy and/or vacancyconditions. For example, when the power source to the visible lightsensor 180 is a battery, the use of the PIR sensor 182 may limit theincreased consumption of power that may be caused by the use of thevisible light sensing circuit.

The visible light sensor 180 may use the PIR sensor 182 to assist inidentifying users and/or movement in one or more different settings. Forexample, the visible light sensor 180 may use the PIR sensor 182 in theroom 102 to determine occupancy during low light conditions. The PIRsensor 182, for example, may be used to identify a user 192 in bed atnight. The visible light sensor 180 may use the PIR sensor 182 if adaylight glare condition prevents and/or decreases the visible lightsensor 180 from identifying an occupancy/vacancy condition.

The visible light sensor 180 may identify the presence of a user 192 andthe PIR sensor 182 may be used to identify the movement of the user 192.For example, the visible light sensor 180 may identify a user gettinginto bed and/or the PIR sensor 182 may identify when the user 192 iswaking from the bed (e.g., based on a movement of the user 192). Thevisible light sensor 180 may control one or more control devices (e.g.,fixtures 172, 174, 176, 178) based on the user getting into bed and/orwaking from bed. For example, the visible light sensor 180 may adjustthe control devices using one or more scenes, such as a wakeup scene. Awakeup scene may include, for example, high energy music being playedand/or the lighting fixtures 172, 174, 176, 178 incrementally increasingthe lighting intensity provided within the room 102.

The visible light sensing circuit of the visible light sensor 180 may bedisabled and the PIR sensor 182 may be enabled after a period of time ofvacancy in a region of interest. When the PIR sensor 182 detects anoccupancy condition in the room 102, the PIR sensor 182 may beconfigured to enable the visible light sensor 180 to generate images andidentify a continued occupancy condition or a vacancy condition. The PIRsensor 182 may enable the visible light sensing circuit of the visiblelight sensor 180 immediately after detecting an occupancy condition inthe room 102. If the visible light sensor 180 is enabled, the visiblelight sensor 180 may be configured to control one or more controldevices based on the occupancy/vacancy condition. For example, thevisible light sensor 180 may be configured to determine a user 192 isoccupying a region of interest and the visible light sensor 180 may beconfigured to provide lighting (e.g., from lighting fixtures 172, 174,176, 178) at the region of interest. The visible light sensor 180 mayuse the PIR sensor 182 to operate similarly to a daylight sensor.Examples of RF load control systems having daylight sensors aredescribed in greater detail in commonly-assigned U.S. Pat. No.8,410,706, issued Apr. 2, 2013, entitled METHOD OF CALIBRATING ADAYLIGHT SENSOR; and U.S. Pat. No. 8,451,116, issued May 28, 2013,entitled WIRELESS BATTERY-POWERED DAYLIGHT SENSOR, the entiredisclosures of which are hereby incorporated by reference.

The visible light sensor 180 may operate in a hospital to configurezones between regions of interest for patients. For example, the visiblelight sensor 180 may identify movement of a patient from a bed to abathroom and may configure the lighting fixtures along the path in thesame zone for lighting control. The zone may be turned on and/orcontrolled to a predefined dimming level when the patient is identifiedas moving in along the path of the defined zone. The visible lightsensor 180 10 may be configured to provide an alert, such as a bedsidealarm (e.g., flashing the lighting fixtures if an immobile patientattempts to get out of a hospital bed).

If the user 192 is exhibiting a sleep condition (e.g., lack of movementfor a predetermined amount of time and/or eyes closed for apredetermined amount of time), the visible light sensor 180 may identifythat the user 192 is in a sleep condition and the visible light sensor180 may transmit a digital message (e.g., including controlinstructions) to the lighting control devices to reduce the dimminglevel or turn off the lights. If the user 192 is exhibiting a sleepcondition, the motorized window treatments 150 may be lowered to apredefined level or to a fully closed position. If the user 192 isexhibiting a sleep condition, the thermostat 160 may be lowered. If theuser 192 is exhibiting a sleep condition, the color temperature of thelighting fixtures 172, 174, 176, 178 may be changed to a warmer (e.g.,redder) color temperature to help the user 192 go to sleep, or the colortemperature of the lighting fixtures 172, 174, 176, 178 may be changedto a cooler (e.g., bluer) color temperature to help the user 192 stayawake and productive.

If the user 192 is exhibiting an alert condition (e.g., the user 192 ismoving for a predefined amount of time and/or the user's eyes are openfor a predetermined amount of time), the visible light sensor 180 maydetermine that the user is in an awake condition and the visible lightsensor 180 may transmit a digital message to the lighting controldevices to increase the dimming level or turn on the lights. If the user192 is exhibiting an alert condition, the motorized window treatments150 may be raised to a predefined level or to a fully opened position,the thermostat 160 may be increased, and/or the color temperature of thelighting fixtures 172, 174, 176, 178 may be changed to a cooler (e.g.,bluer) color temperature to help the user 192 stay awake and productive.

The visible light sensor 180 may determine an emergency condition andmay control the load control devices in response to the emergencycondition. For example, the visible light sensor 180 may identify a userperforming an emergency gesture (e.g., waving hands in a predefinedmanner, moving a mouth in a predefined manner, etc.). The visible lightsensor 180 may identify an emergency condition (e.g., the user havingfallen, the user bleeding, the user not breathing, etc.). The visiblelight sensor 180 may determine that there is in an emergency conditionusing the user's gesture and/or the user's condition. The visible lightsensor 180 may transmit a digital message (e.g., including controlinstructions) to one or more load control devices to provide anemergency signal. For example, the visible light sensor 180 may send asignal to the lighting fixtures 172, 174, 176, 178 to flash on and offor change the color temperature of the lighting fixtures 172, 174, 176,178 if an emergency condition is detected. The visible light sensor 180may be configured to send a digital message to caregivers and/or tonotify emergency personnel based on the detection of an emergencycondition.

The visible light sensor 180 may be configured to generate images thatidentify individual users. The visible light sensor 180 may identify auser entering, exiting, performing a task, and/or residing within theroom 102 using one or more recognition techniques, such as facialrecognition, gait recognition, body-type recognition, and/or anotherimage recognition technique. For example, the visible light sensor 180may be configured to identify a user using the user's facial featuresidentified in the generated images. The visible light sensor 180 may beconfigured to identify a user using a feature-based approach to facialrecognition. In the feature-based approach, the visible light sensor 180may analyze the image to identify, extract, and/or measure facialfeatures (e.g., distinctive facial features) of the user. For example,the visible light sensor 180 may analyze the image to identify, extract,and/or measure the eyes, mouth, nose, etc., of a user. Using the facialfeatures identified, extracted, and/or measured, the visible lightsensor 180 may be configured to compute one or more geometricrelationships among the facial features. By computing the geometricrelationships among the facial features, the facial features may beconverted to a vector of geometric features. Statistical patternrecognition techniques may be employed to match faces using thegeometric features. The visible light sensor 180 may also, oralternatively, be configured to identify a user by the speed of theuser's gait and/or the length of the user's gait. The visible lightsensor may be configured to identify a user who is entering, exiting,performing a task, and/or residing within the room 102.

The visible light sensor 180 may be mounted in one or more locations(e.g., on a wall) and/or orientations to provide an ability to identifya user. For example, a visible light sensor 180 mounted on the wall ofthe room 102 may be in a better position to identify a user using facialrecognition than a visible light sensor 180 that is mounted on theceiling. One or more visible light sensor 180 devices may be usedtogether to provide a composite identification of a user. For example, awall mounted visible light sensor 180 may be configured to identify afront profile of a user 192 and a ceiling mounted visible light sensor180 may be configured to identify a top profile of the user 192. Thevisible light sensor 180 mounted on the ceiling may better identify thegait of the user. The visible light sensor 180 may be configured tocombine the front profile and the top profile of the user to create acomposite profile of the user.

The visible light sensor 180 may be configured to control one or moreload control devices based the identity of the user 192. For example,the user 192 may desire that a region of interest (e.g., the user's desk106 area) be provided with a predefined intensity of light, a predefinedcolor temperature of light, and/or a predefined temperature. The visiblelight sensor 180 may identify when the user 192 enters and/or exits theroom 102. The visible light sensor 180 may adjust the lighting fixtures172, 174, 176, 178 to predefined light intensities and/or colortemperatures, using the identity of the user 192. The visible lightsensor 180 may adjust the HVAC 162 to a predefined temperature and/orthe covering material 152 of the motorized window treatments 150 topredefined settings, using the identity of the user 192 entering,exiting, performing a task, and/or residing in room 102. For example,the visible light sensor 180 may adjust the load control devices to anenergy saving setting and/or another preferred setting when the room 102is vacant.

The visible light sensor 180 may be configured to operate in adaylighting sensor mode. For example, the visible light sensor 180 maybe configured to identify an amount of total lighting intensity in theareas of the room 102 controlled by the lighting fixtures 172, 174, 176,178 and control the dimming level of each of the lighting fixtures 172,174, 176, 178 to maintain an overall lighting level in the room 102. Thetotal lighting intensity may be determined using one or more algorithmor image analysis techniques. For example, the total lighting intensitymay be determined from a red, green, and blue (RGB) image. Using the RGBimage, the total lighting intensity may be defined as(0.299*R+0.587*G+0.114*B).

The visible light sensor may determine the total lighting intensity ofthe room 102, or areas within the room 102 (e.g., including artificiallight provided by one or more of the lighting control devices locatedwithin the room 102 and/or the light provided by the sunlight 196) andadjust the dimming level of the lighting fixtures 172, 174, 176, 178 toenable an overall lighting level to be obtained. The visible lightsensor 180 may determine whether the intensity of light in the room 102is uniform and adjust the dimming level of one or more of the lightingfixtures 172, 174, 176, 178 to obtain a uniform total lighting level inthe room 102. The intensity of light provided by the lighting fixtures172, 174, 176, 178 may not be uniform for one or more reasons, such asimproper settings of the lighting control devices, sunlight 196 enteringthe room 102, one or more of the lighting fixtures 172, 174, 176, 178being in improper working order, etc. The visible light sensor 180and/or the system controller 110 may be configured to transmit an RFsignal to the lighting fixtures 172, 174, 176, 178 to provide apreferred and/or recommended lighting intensity within the room 102. Forexample, the visible light sensor 180 may be configured to transmit RFsignals 108 to provide a uniform lighting intensity throughout the room102.

The visible light sensor 180 may transmit digital messages (e.g.,including the measured lighting intensity) via the RF signals 108. Forexample, the visible light sensor 180 may transmit digital messages(e.g., indications of environmental characteristics, from which controlinstructions may be generated) to the system controller 110 forcontrolling the intensities of the lighting fixtures 172, 174, 176, 178in response to the measured lighting intensity. The measured lightingintensity may be identified in different portions of the room 102 basedon a relative difference in lighting level identified in the generatedimages. The visible light sensor 180 may identify the lighting intensity(luminance) in one or more portions of the room 102 by performing anintegration technique. For example, the visible light sensor 180 mayintegrate across a portion of the room 102. The visible light sensor 180may identify the relative difference in lighting level in the image bydetermining the lighting intensity (luminance) of the portions of theroom 102 and averaging the lighting intensity (luminance) of one or moreof the pixels in the selected portions.

The visible light sensor 180 may identify portions of the generatedimages that include reflected light that is brighter than other portionsof the room by a predefined amount. For example, the visible lightsensor 180 may identify the sunlight 196 entering the room 102 and dimthe lighting fixtures 174, 178 and/or a LED light source on the portionof the room 102 that is affected by the sunlight 196. The visible lightsensor 180 may be configured to focus on one or more regions of interestin the image recorded by the camera and measure the relative differencein lighting intensity in each of the different regions of interestgenerated by the images.

The visible light sensor 180 may be configured to disregard one or moreregions of interest in the image recorded by the camera so that lightingintensity in each of the disregarded regions of interest are notconsidered when performing control. The visible light sensor 180 mayadjust the control parameters (e.g., gain) of light sources, based onthe region of interest in which the lighting intensity is presentlybeing measured. The visible light sensor 180 may transmit digitalmessages (e.g., including control instructions) to the system controller110 for adjusting the lighting fixtures 172, 174, 76, 178 to a preferredlighting intensity, and/or the visible light sensor 180 may transmitdigital messages to the system controller 110 for adjusting the lightingfixtures 172, 174, 76, 178 to a uniform lighting intensity.

The fixtures within the different regions of interest may be identifiedby the location of the lighting levels within the image. Differentfixtures may be mapped to different portions of the images generatedfrom the visible light sensor 180. The dimming level in the differentregions of interest may be adjusted by a predefined amount or may beadjusted to different dimming levels based on the difference in lightinglevels identified between the different regions. For example, thevisible lighting sensor 180 and/or the system controller 110 may changethe dimming level of the lighting fixtures 174, 178 by 25% when theportion of the room 102 that includes the sunlight 196 is determined tobe 25% brighter. Examples of RF load control systems having daylightsensors are described in greater detail in commonly-assigned U.S. Pat.No. 8,410,706, issued Apr. 2, 2013, entitled METHOD OF CALIBRATING ADAYLIGHT SENSOR; and U.S. Pat. No. 8,451,116, issued May 28, 2013,entitled WIRELESS BATTERY-POWERED DAYLIGHT SENSOR, the entiredisclosures of which are hereby incorporated by reference.

The visible light sensor may determine a baseline amount of total light(e.g., artificial light and/or sunlight) present within the room 102, orportions thereof, by analyzing the brightness of generated images. Thebaselines may allow the visible light sensor 180 to identify changes inlighting levels within the room 102 for enabling control according tothe identified lighting levels. The baseline amount of light may operateduring the daylighting sensor mode and/or the daylight glare sensormode.

The baseline amount of light may be a zero light level, a full lightlevel, and/or a number of interval light levels that may fall betweenzero light level and full light level. For example, the visible lightsensor 180 may determine a baseline having zero light by recording animage of the room 102 with the lighting loads in an off state andsunlight being absent from the room 102 (e.g., at nighttime). Thevisible light sensor 180 may determine a baseline having zero artificiallight by recording an image of the room 102 with the lighting loads inan off state and the covering material 152 of the motorized windowtreatments 150 in a fully closed state (e.g., during the day). Thevisible light sensor 180 may determine a baseline having a fullartificial light level by recording an image of the room 102 at a timewhen one or more of the lighting loads are turned on to their fulldimming level (e.g., 100% intensity). The visible light sensor 180 maydetermine a baseline at a full light level by recording an image of theroom 102 at a time when one or more of the lighting loads are turned onto their full dimming level (e.g., 100% intensity) and when the coveringmaterial 152 of the motorized window treatments 150 are in a closedstate and/or open state.

Baseline intervals (e.g., 10%, 20%, 30%, etc., intensities) ofartificial light within the room 102 may be provided using one or morecombinations of on states of lighting loads within room 102. The visiblelight sensor 180 may determine baselines having different baselineintervals by recording an image of the room 102 at times in which one ormore of the lighting loads are turned on to the respective intervals(e.g., 10%, 20%, 30%, etc., intensities) and when the covering material152 of the motorized window treatments 150 are in a closed state and/oran open state.

Baseline intervals of artificial light may be provided within one ormore regions of interest. For example, baseline intervals may beprovided on a user's task area (e.g., a user's desk 106) and/or otherregions of interest. The visible light sensor 180 may record one or moreimages of the room 102 while light sources cycle through lightingintensity levels. For example, the visible light sensor 180 may recordimages of the room as one or more of lighting fixtures 172, 174, 176,178 cycle through an increasing lighting intensity of 10%, 25%, 30%,50%, etc. The lighting fixtures 172, 174, 176, 178 may receive a command(e.g., including control instructions) from the system controller 110 tocycle through the dimming levels (e.g., resting on each dimming levelfor a period of time or increasing dimming levels over a period oftime), or may receive commands (e.g., including control instructions)from the system controller 110 to change from each dimming level. Thecommands (e.g., including control instructions) may be triggered fromthe mobile device 190.

The visible light sensor 180 may record an image of the room 102 anddetermine whether the image of the room 102 is equivalent to one or morebaseline images of the room 102. The baseline images may be used todetermine target intensity levels, target color levels, color shifts,daylight contribution, etc. For example, the visible light sensor 180may determine if an amount of light presented within the room 102 isequal to a baseline amount of light recorded within the room 102. If theamount of light is different than a baseline, the visible light sensor180 may identify the baseline that is the closest to the currentlighting level.

The visible light sensor 180 may determine if the light within the room102 differs from a preferred baseline amount of light within the room102 by comparing the amount of light within the room 102 with thebaseline amount of light within the room. The visible light sensor 180may control the lighting loads (e.g., lighting fixtures 172, 174, 176,178) and/or the motorized window treatments 150 to achieve a preferredlighting level that is equivalent to, or greater than, a previouslyrecorded baseline.

The visible light sensor 180 may equate the baseline lighting intensitywith a preferred lighting intensity within a space. The preferredartificial lighting intensity may be a lighting intensity defined by auser. For example, the user 192 may prefer a predefined lightingintensity at the users' task area (e.g., a desk 106, a monitor 166, apredefined area around the user 192). The visible light sensor 180 mayrecord an image of the user's task area and determine whether apreferred amount of light is provided to the user's task area. Thevisible light sensor 180 may determine whether a preferred amount oflight is provided to the user's task area by comparing the lightingintensity to a baseline lighting intensity identified as the preferredlighting intensity within the space. If an undesired amount of lightingintensity is provided to the user's task area, the visible light sensor180 may send a digital message (e.g., including control instructions) toa lighting control device (e.g., light fixtures 174, 178) or motorizedwindow treatments 150 to provide additional or lesser lighting to theuser's task area.

The visible light sensor 180 may determine a baseline amount of sunlightpresented within a space, using an image. The baseline amount ofsunlight may be zero sunlight intensity, full sunlight intensity, and/ora number of intervals that may fall between zero sunlight intensity andfull sunlight intensity. For example, the visible light sensor 180 maydetermine a baseline having zero sunlight intensity by recording animage of the room 102 when sunlight is absent (e.g., at nighttime)and/or the visible light sensor 180 may determine a baseline having zerosunlight by recording an image when the covering material 152 of themotorized window treatments 150 are in a fully closed state. The visiblelight sensor 180 may determine a baseline having full sunlight intensityby recording an image of the room 102 at a time during the day in whichsunlight is predicted to be at a full potential (e.g., using a time ofday, time of year, location of the building, direction of the windows104, position of the sun in the sky, weather conditions, etc.) and/orwhen the covering material 152 of the motorized window treatments 150are in an open state.

The visible light sensor 180 may be configured to determine baselineintervals (e.g., 10%, 20%, 30%, etc.) of sunlight within the room 102.Each interval may be detected within the images generated by the visiblelight sensor by identifying incrementally brighter images as sunlightenters the room 102. Baseline intervals of sunlight within the room 102may be provided using one or more combinations of secondary conditionsthat may affect the presence of sunlight (e.g., a time of day, time ofyear, location of the building, direction of the windows 104, positionof the sun in the sky, weather conditions, etc.). Also, oralternatively, baseline intervals of sunlight within the room 102 may beprovided using one or more positions of the covering material 152 of themotorized window treatments 150. For example, baseline intervals may beprovided in the room 102 at a time of the day in which the sun ispredicted to provide full sunlight and/or at which the covering material152 of the motorized window treatments 150 are closed a predefinedamount. The visible light sensor 180 may record one or more image of theroom 102 during times of different sunlight strengths and/or using thecovering material 152 of the motorized window treatments 150 beingopened to different amounts (e.g., opened to 10%, 30%, 50%, 70%, 90%capacity).

The visible light sensor 180 may record an image of the room 102 anddetermine whether the image of the room 102 is equivalent to one or morebaseline images of the room 102. The baseline image of the room 102 mayrelate to a baseline lighting intensity present within one or moreregions of interest within the room 102. For example, the visible lightsensor 180 may determine if an amount of sunlight present within one ormore regions of interest within the room 102 differs from a baselineamount of sunlight presented at the one or more regions of interestwithin the room 102. The visible light sensor 180 may determine if thesunlight presented within the room 102 differs from the previouslycaptured baseline amount of sunlight presented within the room 102 bycomparing the images. For example, the visible light sensor 180 maydetermine whether the sunlight in one or more regions of interest withinthe room 102 has increased, decreased, or stayed the same from apreviously captured amount of sunlight presented within the room 102.

The visible light sensor 180 may control the motorized window treatments150 so that the amount of sunlight 196 present within one or moreregions of interest within the room 102 is a preferred amount ofsunlight presented within the one or more regions of interest. Thevisible light sensor 180 may control the motorized window treatments 150so that the amount of sunlight 196 present within one or more regions ofinterest within the room 102 is the same, or similar to, a baseline thatis stored as having the preferred amount of sunlight. The baseline maybe defined during configuration of the load control system 100 and/orupdated during (e.g., on a daily, monthly, etc., basis). The visiblelight sensor 180 may send digital messages (e.g., including controlinstructions) to the motorized window treatments 150 to adjust thecovering material 152 of the motorized window treatments 150 until thepreferred amount of sunlight is reached. During adjustment of thecovering material 152, the visible light sensor 180 may record images ofthe room 102 to identify the amount of sunlight present within the oneor more regions of interest and the digital messages may continue to betransmitted to continue adjusting the covering material 152, or thecovering material 152 may continue to be adjusted until receiving adigital message to stop adjustment.

The visible light sensor 180 may equate the baseline lighting intensitywith a preferred lighting intensity of a light source within the room102. The preferred lighting intensity may be a lighting intensitydefined by a user, such as user 192. For example, the user may desirethat sunlight be minimized within the room 102 (e.g., due to aheightened sensitivity to sunlight and/or a heightened privacyexpectation). That is, the preferred lighting intensity and/or colortemperature may be a lighting intensity and/or color temperature withminimized sunlight, e.g., with the covering material 152 of themotorized window treatments 150 being fully closed. The user may desirethat a predefined amount of lighting intensity be present within theroom 102 and may set the predefined amount of lighting intensity to abaseline lighting intensity.

The visible light sensor 180 may determine if a total lighting intensitypresent within the room 102 differs from the baseline lighting intensityby comparing the total lighting intensity present within the room 102with the baseline lighting intensity. The visible light sensor 180 maycontrol the covering material 152 of the motorized window treatments 150so that the sunlight provided to the room 102 is equivalent to theuser's preferred amount of sunlight. The dimming level of the lightingfixtures 172, 174, 176, 178 may be set to achieve a preferred totallighting intensity in the room 102. The amount of power used by thelighting fixtures 172, 174, 176, 178 may be reduced by allowing agreater level of sunlight into the room 102. The daylight glare may beminimized by reducing the level of the covering material 152 of themotorized window treatments 150 and the reduced lighting intensity maybe compensated for by the light provided by the lighting fixtures 172,174, 176, 178.

The visible light sensor 180 may identify a region of interest in whichlighting intensity resulting from sunlight 196 may meet the lightingintensity resulting from other sources (e.g., lighting fixtures 172,174, 176, 178). For example, the visible light sensor 180 may identify apoint and/or a line at which sunlight may cease to enter the room 102(e.g., due to the level of the covering material 152 on the motorizedwindow treatments 150). The visible light sensor 180 may be configuredto adaptively determine whether lighting intensity resulting fromsunlight presented at a region of interest (e.g., task area) is apreferred intensity of light. The visible light sensor 180 may beconfigured to control one or more devices based on whether the lightingintensity resulting from sunlight present at the region of interest(e.g., task area) is a preferred intensity of light. For example, thevisible light sensor 180 may be configured to determine the location ofa user's task area and the preferred lighting intensity resulting fromsunlight at the task area. The visible light sensor 180 may adjust thecovering material 152 on the motorized window treatments 150 so that thelighting intensity resulting from sunlight presented on the user's taskarea is similar to and/or equivalent to the preferred intensity ofsunlight at the task area.

A user may desire that sunlight be present within the room 102, but thatthe sunlight be prevented from one or more regions of interest withinthe room 102. For example, a user may desire that sunlight be presentwithin the room 102 but the sunlight prevented from a user area (e.g.,desk 106) of the room 102. The visible light sensor 180 may control themotorized window treatments 150 to determine the level at which thecovering material 152 may be opened to provide sunlight at one or moreregions of interest and prevent sunlight at one or more regions ofinterest. The visible light sensor 180 may consider secondary conditions(e.g., a time of day, time of year, location of the building, directionof the windows 104, position of the sun in the sky, weather conditions,etc.) in determining the amount of which the covering material 152 maybe opened to provide sunlight at one or more regions of interest andprevent sunlight at one or more regions of interest. For example, thevisible light sensor 180 may determine that on a cloudy day, in June, at1:00 p.m., the covering material 152 of the motorized window treatments150 should be closed 40% so that sunlight 196 is prevented from reachingthe desk 106. The visible light sensor 180 may determine whether a usertask area (e.g., desk 106, monitor 166, predefined area around the user192, etc.) is moved and the visible light sensor 180 may control themotorized window treatments 150 so that sunlight 196 is prevented fromreaching the user task area.

The visible light sensor 180 may be configured to adjust one or morelighting fixtures 172, 174, 176, 178 so that the lighting intensitypresent at a region of interest is equal to a preferred or recommendedlighting intensity. The preferred lighting intensity may be defined by auser 192. The recommended lighting intensity may be defined by themanufacturer or lighting designer. The preferred lighting intensityand/or the recommended lighting intensity may be defined when the loadcontrol system 100 is being configured and may be updated duringoperation.

The visible light sensor 180 may define the preferred lighting intensityto be used within the room 102 after identifying the user. For example,the visible light sensor 180 may identify the user 192 via an image ofthe user 192 within the room 102, via a mobile device 190 used by theuser 102 within the room 102, and/or using another identificationprocedure (e.g., via audio identification, login identification, etc.).The visible light sensor 180 may be configured to transmit a digitalmessage to the lighting fixtures 172, 174, 176, 178 so that the lightingfixtures 172, 174, 176, 178 may present light intensities thatcorrespond to the preferred lighting intensity of the user 192. Forexample, the visible light sensor 180 may be configured to transmit adigital message to lighting fixture 174 and lighting fixture 178 toprovide additional lighting to the user's keyboard 168 and/or desk 106,if such lighting is preferred by the user 192.

The visible light sensor 180 may be configured to adjust lightingsources to compensate for additional or deficient artificial lighting ornatural lighting (e.g., sunlight). For example, an increased lightingintensity, such as by sunlight 196, may be provided by a window 104. Tocompensate for the increased lighting intensity provided by the window104, the visible light sensor 180 may be configured to adjust lightingloads within a predefined distance of the window 104 (e.g., during thedaylighting sensor mode). For example, the visible light sensor 180 maybe configured to send an RF signal to lighting fixtures 174, 178 toprovide less artificial lighting, based on an increased amount ofsunlight provided by the window 104. Also, or alternatively, the visiblelight sensor 180 may be configured to send an RF signal to lightingfixtures 172, 176 to provide additional lighting (e.g., during thedaylighting sensor mode). The amount of additional lighting may be basedon the amount of sunlight provided by the window 104, such that thetotal light may reach a baseline lighting level or higher.

The preferred or recommended lighting intensity (luminance) may berecorded in an image by visible light sensor 180 (e.g., with or withoutdaylight). The visible light sensor 180 may identify when the lightinglevel (luminance) is above or below the preferred or recommendedlighting intensity by an identified amount. The lighting level(luminance) may be identified by comparing a previously recorded imageof the room 102 with the current image of the room. The lightingfixtures 172, 174, 176, 178 and/or the motorized window treatments 150may be controlled to meet the preferred or recommended lightingintensity (luminance). The motorized window treatments 150 may beadjusted prior to the lighting fixtures to avoid additional energyconsumption. If the sunlight 196 allowed by the motorized windowtreatments 150 does not meet the preferred or recommended lightingintensity (luminance), the lighting intensity of the lighting fixtures172, 174, 176, 178 may be increased or decreased. When the preferred orrecommended lighting intensity (luminance) is identified in the imagesgenerated by the visible light sensor 180, the load control may ceasefor a predefined period of time or until the lighting level is againabove or below the preferred or recommended lighting intensity(luminance). The visible light sensor 180 may be configured to adjustthe lighting at the task area to a lighting intensity other than thepreferred or recommended lighting intensity (luminance) when the user192 leaves the task area (e.g., for a predefined period of time). Thelighting at the task area may be reduced to a predefined amount, orturned off, to avoid or reduce energy usages when the user is absentfrom the task area.

An increased and/or decreased lighting intensity may be presented at afirst region of interest and at a second region of interest depending onone or more characteristics of the light source. For example, the firstregion of interest and the second region of interest may have adifferent lighting intensity depending on the age, model, size,operability, etc., of the light sources within the first region ofinterest and the second region of interest. The visible light sensor 180may determine the lighting intensity presented at a region of interestto determine if the dimming level of the light source should be adjustedwithin the region of interest. For example, the lighting fixtures 172,176 may be illuminating at a level above or below a preferred lightingintensity within the second region of interest. The lighting fixtures172, 176 may be illuminating at a different lighting intensity due tothe age and/or operability of the lighting fixtures 172, 176. Thevisible light sensor 180 may be configured to adjust the lightingfixtures 172, 176 to increase or decrease the lighting intensityprovided by the lighting fixtures, to compensate for the less thanpreferred lighting intensity (e.g., increase the dimming level due to adecreased intensity caused by aging).

The visible light sensor 180 may adjust one or more light sources (e.g.,fixtures 174, 178) to provide a uniform lighting intensity withindifferent regions of interest. The different regions of interest may beidentified by the relatively different levels of reflected light in eachportion of the images generated of the room 102. The visible lightsensor 180 may identify the relative difference in lighting level in theimage by determining the lighting intensity (luminance) of the regionsof interest of the room 102 and averaging the lighting intensity(luminance) of one or more of the pixels in the selected regions ofinterest. Sunlight 196 may enter windows 104 and may be presented withina first region of interest and a second region of interest may beunaffected. For example, the depth of sunlight may be identified in thefirst region of interest, but not the second region of interest. Thefirst region of interest may be provided with a lower artificiallighting intensity level than the second region of interest. To accountfor the sunlight 196 being presented in the first region of interest andnot in the second region of interest, the fixtures within the firstregion of interest (e.g., lighting fixture 174, 178) may be adjusted toa lower dimming level in that region of interest than the fixtureswithin the second region of interest (e.g. lighting fixtures 172, 176).

The visible light sensor 180 may determine whether one or more regionsof interest are presenting a uniform lighting intensity. For example,the first region of interest may have a lighting intensity of artificiallight that is higher than a lighting intensity of artificial lightprovided at the second region of interest. Other devices within a regionof interest may also be providing additional light in a region ofinterest. For example, the lamp 142 may be providing light in the regionbeing lit by the lighting fixture 176. A peripheral device (e.g., amonitor 166) may be illuminating a region of interest. The motorizedwindow treatments 150 may be providing daylight in a region of interest.The regions of interest may be sub-areas of an object within the room102, such as sub-areas of the desk 106 that have different lightinglevels.

The visible light sensor 180 may adjust the lighting intensity of one ormore of the lighting fixtures 172, 174, 176, 178 to uniformly illuminateone or more regions of interest within the room 102. For example, thevisible light sensor 180 may adjust the lighting intensity of lightfixtures 174, 178 so that lighting fixtures 174, 178 illuminatesuniformly with the light intensities provided by lighting fixtures 172,176. The visible light sensor 180 may adjust the lighting intensity oflighting fixtures 174, 178, for example, to account for the age and/oroperation of the lighting fixtures 174, 178.

The visible light sensor 180 may adjust the lighting sources tocompensate for intensities of light (e.g., daylight and/or artificiallight) at a first region of interest that are higher and/or lower thanintensities of light (e.g., daylight and/or artificial light) that areprovided at the second region of interest. For example, a user's desk106 may receive additional lighting 198 from a computer monitor 160. Theadditional lighting 198 may result in a sub-area of the desk 106 havinga lighting intensity that is greater than lighting intensity provided onthe remaining sub-areas of the desk 106. The additional lighting 198 mayresult in a sub-area of the desk 106 having a greater lighting levelthan preferred by the user 192 and/or otherwise desired (e.g.,recommended by the lighting manufacturer). The visible light sensor 180may send a digital signal to one or more lighting fixtures 172, 174, 76,178 to reduce lighting at the location of the user's desk 106 thatprovided the additional lighting 198. For example, the visible lightsensor 180 may reduce lighting at a defined portion of the user's desk106 receiving the additional lighting 198 by transmitting an RF signalto control light source within a predefined distance of the definedportion of the user's desk 106, or light sources having the greatestinfluence on the illuminance distribution on the defined portion of theuser's desk 106, so that the light source reduces the light provided tothe defined portion of the user's desk 106. By reducing light to theuser's desk 106 receiving the additional lighting 198, the sub-areareceiving the additional light 198 by the computer monitor 166 may beuniform to sub-areas not receiving light via the computer monitor 166.

The visible light sensor 180 may transmit a message to the user 192notifying that one or more lighting sources are providing less thanpreferred or recommended lighting intensities. The visible light sensor180 may transmit a message to the user (e.g., via the mobile device 190)notifying that one or more of the lighting fixtures 172, 174, 176, 178has been adjusted, or may be adjusted, to compensate for the less thanpreferred or recommended lighting intensity. The visible light sensor180 may transmit a message to the user (e.g., via the mobile device 190)indicating the identities of the lighting fixtures 172, 174, 176, 178for which compensation was provided or is recommended to be provided.The visible light sensor 180 may indicate to the user (e.g., via themobile device 190) the remaining life (e.g., in hours, days, etc.) thatthe lighting fixtures 172, 174, 176, 178 have remaining before theyshould be replaced. The original life of a light source may be stored atthe visible light sensor 180 upon receiving an indication ofinstallation of the light source, and the visible light sensor 180 maycount down from the original life using a timeclock.

As described herein, the visible light sensor 180 may be configured todisregard movement, lighting intensity (e.g., lighting intensity fromsunlight and/or artificial light), color temperature, occupancy/vacancyconditions, etc. at a region of interest when performing lightingcontrol. A portion of the room 102 may be separately masked from anotherportion of the room 102. Masking a portion of the room 102 may result inthe visible light sensor 180 disregarding the portion of the room 102.For example, the visible light sensor 180 may disregard movement,lighting intensity (e.g., lighting intensity from sunlight and/orartificial light), color temperature, occupancy/vacancy conditions atthe door 105, which may be irrelevant to the operation of the loadcontrol devices within the room 102. The visible light sensor 180 may beconfigured to discriminate between different colors of light presentedat different regions of interest when performing lighting control.

The visible light sensor 180 may be configured to operate in a colorsensor mode. In the color sensor mode, the visible light sensor 180 maybe configured to determine a color temperature displayed within theimages of the room 102. The color temperature of a light source mayrefer to the temperature of an ideal black body radiator that radiateslight of comparable hue to that of the light source. For example,candlelight, tungsten light (e.g., from an incandescent bulb), earlysunrise, and/or household light bulbs may appear to have relatively lowcolor temperatures, for example on the range of 1,000-3,000 degreesKelvin. Noon daylight, direct sun (e.g., sunlight above the atmosphere),and/or electronic flash bulbs may appear to have color temperaturevalues on the order of 4,000-5,000 degrees Kelvin and may have agreenish blue hue. An overcast day may appear to have a colortemperature of approximately 7,000 degrees Kelvin and may be even bluerthan noon daylight. North light may be bluer still, appearing to have acolor temperature on the range of 10,000 degrees Kelvin. Colortemperatures over 5,000 degrees Kelvin are often referred to as coolcolors (e.g., bluish white to deep blue), while lower color temperatures(e.g., 2,700-3,000 degrees Kelvin) are often referred to as warm colors(e.g., red through yellowish white).

The visible light sensor 180 may be configured to sense a color (e.g.,measure a color temperature) of light emitted by one or more of thelighting fixtures 172, 174, 176, 178 in the room 102. For example, thevisible light sensor 180 may be configured to operate as a color sensoror a color temperature sensor. The visible light sensor 180 may beconfigured to measure a color temperature within one or more regions ofinterest within the room 102 based on the color temperature presentedwithin the room 102.

The visible light sensor 180 may be configured to measure a colortemperature of one or more regions of interest within the room 102 usinga color wheel. The color wheel may be configured to display one or morecolors. For example, the color wheel may include standard RGB colors.The visible light sensor 180 may be configured to record an image of thecolor wheel and/or measure a color temperature of a light emitted by oneor more of the lighting fixtures 172, 174, 176, 178 using the colorwheel. The colors on the color wheel may be identified in the generatedimages of the room 102. A relative difference in color temperature fromthe colors on the color wheel may be identified in the reflected lightcaptured in the generated images.

In the color sensor mode, the visible light sensor may transmit digitalmessages (e.g., including indications of the identified colortemperature or control instructions based on the identified colortemperature) to one or more of the lighting fixtures 172, 174, 176, 178to control the color (e.g., the color temperatures) provided by thelighting fixtures 172, 174, 176, 178. The visible light sensor 180 maytransmit the digital message to the lighting fixtures 172, 174, 176, 178via RF signals, such as the RF signals 108. The visible light sensor 180may transmit the digital message to the lighting fixtures 172, 174, 176,178 in response to the identified lighting intensity (e.g., color tuningof the light in the room).

The visible light sensor 180 may identify the portion of the room 102occupied by the sunlight 196 as being of a color temperature that isrelatively more red than the other portion of the room 102. The visiblelight sensor 180 may change the color temperature of the lightingfixtures 174, 178 to a different color temperature than the lightingfixtures 172, 176 in response to the identification of the colortemperature of the sunlight 196 in a portion of the room. The colortemperature of the lighting fixtures 174, 178 may be changed to arelatively cooler (e.g., bluer) color than that of the lighting fixtures172, 176 on the interior of the room to reduce the gradient of the colortemperature in the room caused by the sunlight 196. The colortemperature of the lighting fixtures 172, 176 may be changed to arelatively redder color temperature to reduce the gradient of the colorof the sunlight 196 in the portion of the room affected by the sunlight196. The color temperature of the lighting fixtures 172, 174, 176, 178may change from relatively redder color temperatures from the directionof the windows 104, or to relatively bluer color temperatures in thedirection of the windows 104, to reduce the gradient in colortemperature caused by the sunlight 196. An example of a load controlsystem for controlling the color temperatures of one or more lightingloads is described in greater detail in commonly-assigned U.S. PatentApplication Publication No. 2014/0312777, published Oct. 23, 2014,entitled SYSTEMS AND METHODS FOR CONTROLLING COLOR TEMPERATURE, theentire disclosure of which is hereby incorporated by reference.

The visible light sensor 180 may identify baseline color temperaturespresent within one or more regions of interest within a space (e.g.,room 102) within generated images. The baseline color temperatures maybe identified in images representing relatively different colortemperatures within the room 102. The images that include the baselinecolor temperatures may range across the color spectrum (e.g., whitecolor spectrum) from a relatively bluer color temperature to arelatively redder color temperature.

The visible light sensor 180 may determine baseline color temperaturesby recording images of the room 102 by controlling the lighting loads toa relatively bluer color temperature on the color spectrum and changingthe color temperature of the lighting loads to a relatively redder colortemperature, or vice versa, and recording images at different colortemperatures. For example, color temperatures over 5,000 degrees Kelvinmay be referred to as cool colors (e.g., a bluish white color), whilecolor temperatures from 2700-3000 degrees Kelvin may be referred to aswarm colors (e.g., yellowish white through red). The color temperaturesmay be incremented by a predefined amount to create baseline images ateach predefined amount. The images may be recorded at a time whendaylight or other ambient light is minimized in the space. For example,the images may be recorded at nighttime and/or when the coveringmaterial 152 of the motorized window treatments 150 are in a closedstate to prevent sunlight from affecting the baseline colortemperatures.

The color temperature may be determined by measuring a region (e.g., oneor more pixels) where a pure white card may be placed. The visible lightsensor 180 may generate images of the changing color temperature acrossthe spectrum (e.g., white color spectrum) and store the images asbaseline images for color temperature. For example, the visible lightsensor 180 may identify the changes in color temperature across thespectrum by a certain number of degrees Kelvin and store the imagerepresenting the interval change as the next baseline interval for colortemperature. The image may be stored (e.g., stored by the visible lightsensor 180) with the control settings for the color temperature. Theimages may not be stored by the visible light sensor 180 if theintermediate and/or historic color calculations are used.

Baseline color temperatures may be provided within one or more regionsof interest. To determine baseline color temperatures, the visible lightsensor 180 may record one or more images of the room 102 while the lightsources cycle through different colors at the different regions ofinterest. While the visible light sensor 180 analyzes the colors ofimages at one region of interest, other portions of the room 102 may bemasked. For example, the visible light sensor 180 may record images ofthe room as one or more of lighting fixtures 172, 174, 176, 178 cyclethrough different colors across the lighting spectrum (e.g., white lightspectrum). The baseline color temperatures for each region of interestmay be stored at the visible light sensor 180 for identifying a changein color temperature in the regions of interest and/or controlling thelight in the regions of interest to an identified color temperature.

The visible light sensor 180 may determine a lumen depreciation or colorshift in the light output of the light fixtures by comparing a presentcolor temperature of the light sources (e.g., lighting fixtures 172,174, 176, 178) with a baseline color temperature. For example, thevisible light sensor 180 may record an image of the room 102 anddetermine whether color temperature of the room 102 is equivalent to oneor more baseline color temperatures of the room 102. The visible lightsensor 180 may determine the color temperature difference between thepresent image and the baseline image. The visible light sensor 180 maydetermine the lumen depreciation or color shift of the light sourcesbased on the difference of the present color temperature within the room102 and the baseline color temperature presented within the room 102.

The visible light sensor 180 may operate in a daylight glare sensormode. In the daylight glare sensor mode, the visible light sensor 180may be operated to decrease or eliminate the amount of sunlight glarebeing presented in the room 102. The visible light sensor 180 may beconfigured to increase, decrease, or eliminate the amount of sunlightglare that enters the room 102. For example, sunlight or sunlight glaremay be prevented from reaching a task area (e.g., a desk 106) of theuser 192. The visible light sensor 180 may identify the user task area(e.g., the desk 106) of the user 192, identify sunlight glare in thegenerated images, and decrease and/or eliminate the amount of sunlightglare presented at the user's task area. For example, the visible lightsensor 180 may be configured to transmit a digital message (e.g.,including indications of environmental characteristics or controlinstructions) to the system controller 110 and/or the motorized windowtreatments 150 to lower the covering material 152 to a level thatdecreases or eliminates the amount of sunlight glare being presented ata user's task area.

The visible light sensor 180 may identify an amount of sunlight glarewithin a region of interest (e.g., a user's task area, such as desk 106)by determining a depth of sunlight 196 that is entering the room 102.The visible light sensor 180 may determine the depth of sunlight 196penetration into the room 102 from the image recorded by the camera. Thesunlight 196 may be identified as light that is coming from thedirection of the windows 104 and that is relatively brighter (e.g., by apredefined threshold) than the other light in the room 104. The visiblelight sensor 180 may identify direction (e.g., direction within the room102, such as direction of the windows 104) using an electronic compass.The electronic compass may be integrated within the visible light sensor180, or the electronic compass may be external to the visible lightsensor 180. The sunlight 196 may be further identified according to thelocation of the building, the direction of the windows 104 in thebuilding (e.g., whether the windows 104 are facing the direction of thesun), the weather conditions (e.g., a sunny day), the time of day (e.g.,a time of day when the sunlight would be directly penetrating throughthe windows), the time of year (e.g., a time of year when the sunlightwould be directly penetrating through the windows), the position of thesun in the sky, and/or other parameters that may be used to determinethe intensity of sunlight glare at the room 102. The visible lightsensor 180 may use the electronic compass thereon to detect thedirection of other objects, such as the direction of the sun or otherenvironmental characteristics that may be determined based on direction.The visible light sensor 180 may have timeclock thereon to detect thetime of day, time of year, or other parameters regarding time.

The penetration distance of the sunlight 196 into the room may bedetected as the area affected by sunlight glare (e.g., when each of thecharacteristics indicating the sunlight glare is coming through thewindows 104 have been met). When the visible light sensor 180 is locatedon the wall facing the windows 104, the visible light sensor 180 mayidentify the sunlight glare in the images by identifying the orb of thesun through the windows 104.

The visible light sensor 180 may be configured to transmit a digitalmessage (e.g., including control instructions) to the load controldevices (e.g., the motorized window treatments 150) to limit oreliminate the sunlight glare. The visible light sensor 180 may transmita digital message (e.g., including control instructions) to the loadcontrol devices (e.g., the motorized window treatments 150) to limit oreliminate depth of sunlight 196 penetration into the room 102. Forexample, the visible light sensor 180 may be configured to transmit adigital message (e.g., including control instructions) to the loadcontrol devices (e.g., the motorized window treatments 150) to decreaseor eliminate sunlight glare from shining on the user 192 or the user'stask area (e.g., the desk 106, monitor 166, and/or keyboard 168). Thevisible light sensor 180 may be configured to lower the coveringmaterial 152 of each of the motorized window treatments 150 to preventthe depth of sunlight 196 penetration from exceeding a maximum sunlightpenetration depth. Examples of methods for limiting the sunlightpenetration depth in a space are described in greater detail in U.S.Pat. No. 8,288,981, issued Oct. 16, 2012, entitled METHOD OFAUTOMATICALLY CONTROLLING A MOTORIZED WINDOW TREATMENT WHILE MINIMIZINGOCCUPANT DISTRACTIONS, the entire disclosure of which is herebyincorporated by reference.

The visible light sensor 180 may be configured to override or supplementautomated control of the motorized window treatments 150. For example,the visible light sensor 180 may be configured to override or supplementautomated control of the motorized window treatments 150 in response todetermining the time of day, time of year, location of the building,direction of the windows 104, position of the sun in the sky, weatherconditions, etc., that may be used to determine the intensity ofsunlight at the room 102. The weather condition, position of the sun,etc., may be derived from an external device (e.g., an external server,such as a cloud server) and/or the weather condition, position of thesun, etc., may be derived from a window sensor. Examples of load controlsystems having window sensors are described in greater detail in U.S.Patent Application Publication No. 2014/0156079, published Jun. 5, 2014,entitled METHOD OF CONTROLLING A MOTORIZED WINDOW TREATMENT, the entiredisclosure of which is hereby incorporated by reference.

The visible light sensor 180 may operate proactively or reactively tocontrol the motorized window treatments 150. For example, the visiblelight sensor 180 may reactively operate to lower the motorized windowtreatments 150 after identification of the penetration distance of thesunlight 196 on the user 192 or the user's task area. Afteridentification of the sunlight 196 on the user 192 or the user's taskarea, the visible light sensor 180 may send digital messages (e.g.,including control instructions) to close the motorized window treatments150 a predefined distance or until the penetration distance of thesunlight 196 is removed from the user 192 or the user's task area (e.g.,by a predefined distance). The visible light sensor 180 may operateproactively by lowering the motorized window treatments 150 before thepenetration distance of the sunlight 196 on the user 192 or the user'stask area. When the sunlight 196 is identified as being within apredefined distance of the user 192 or the user's task area, the visiblelight sensor 180 may send digital messages (e.g., including controlinstructions) to close the motorized window treatments 150 a predefineddistance or until the penetration distance of the sunlight 196 is withinanother predefined distance from the user 192 or the user's task area.

The visible light sensor 180 may be configured to determine the numberof users within one or more regions of interest (e.g., during anoccupancy/vacancy sensor mode). The visible light sensor 180 may beconfigured to count the number of users entering and/or exiting a regionof interest. For example, the visible light sensor 180 may be configuredto determine that ten users have entered a room 102 and four users haveexited the room 102. The visible light sensor 180 may mask the door andcount the number of users in the room 102, or mask the rest of the room102 and count the number of users that have entered and/or exited theroom 102. Based on the number of users who have entered and/or exitedthe room 102, the visible light sensor 180 may be configured todetermine the number of users remaining in the room 102.

The visible light sensor 180 0 may be configured to control one or moreof the load control devices within the room 102 in response to thenumber of users in the room 102. For example, the visible light sensor180 may be configured to control the HVAC system 162 in response to thenumber of users in the room 102. The visible light sensor 180 may beconfigured to control one or more of the load control devices of theload control system 100 in response to the number of users in the room102 exceeding an occupancy number threshold. For example, the visiblelight sensor 180 may be configured to provide an alert (e.g., flashinglights, changing color of the lights to red) if the number of users inthe room 102 exceeds an undesired and/or unsafe number threshold.

The visible light sensor 180 may increase or decrease the lightingintensity provided by lighting fixtures 172, 174, 176, 178 based on thealertness and/or location of the user 192. The visible light sensor 180may adjust the color temperature provided by lighting fixtures 172, 174,176, 178 based on the alertness and/or location of the user 192. Forexample, the visible light sensor 180 may increase the lightingintensity of the lighting fixtures 172, 174, 176, 178 if the user 192 isat desk 106 and the user's eyes are closed for a predefined period oftime. The visible light sensor 180 may adjust the color temperature ofthe lighting fixtures 172, 174, 176, 178 to a cooler color temperature(e.g., blue) if the user 192 is at desk 106 and the user's eyes areclosed for a predefined period of time. Alternatively, the visible lightsensor 180 may adjust the color temperature of the lighting fixtures172, 174, 176, 178 to a warmer color temperature (e.g., red), viapositive feedback, if the user is in bed and the user's eyes are closedfor a predefined period of time. The lighting intensity and/or colortemperature provided by lighting fixtures 172, 174, 176, 178 based onthe alertness and/or location of the user 192 may be defined by the userand/or may be based on manufacturer settings.

The visible light sensor 180 may adjust the lighting fixtures 172, 174,176, 178 based on the location of the user 192. For example, the visiblelight sensor 180 may increase and/or decrease the lighting intensityprovided by lighting fixtures 174, 178 depending on the location of theuser. For example, the visible light sensor 180 may increase thelighting intensity provided by lighting fixtures 174, 178 if the user192 is within a predefined distance of the lighting fixtures 174, 178.The visible light sensor 180 may decrease the lighting intensityprovided by lighting fixtures 172, 176 if the user is outside of apredefined range of the lighting fixtures 172, 176. The visible lightsensor 180 may adjust the lighting fixtures 172, 174, 176, 178 to emit apredefined color temperature depending on the location of the user. Forexample, the visible light sensor 180 may adjust the lighting fixtures172, 174, 176, 178 to emit a preferred color temperature if the user 192is within a predefined distance of the lighting fixtures 172, 174, 176,178. The visible light sensor 180 may adjust the lighting fixtures 172,174, 176, 178 to emit a color temperature representative of a vacancycondition and/or to save energy if the user is outside of a predefinedrange of the lighting fixtures 172, 174, 176, 178.

The visible light sensor 180 may adjust the load control devices to seta scene, depending on the alertness of the user and/or the location ofthe user. For example, the visible light sensor 180 may set a bed timescene if the user is in bed and the user is exhibiting a sleep condition(e.g., the user's eyes are closed for a predefined time). The bedtimescene may include reducing the light intensities of the lightingfixtures, playing soft music, and/or lowing the covering material 152 ofthe motorized window treatments 150. The bed and/or areas within apredefined distance of the bed may be masked. For example, based onprivacy settings defined by the user 192, the bed and/or areas within apredefined distance of the bed may be masked.

The visible light sensor 180 may set a wake-up scene depending on a timeof day, the location of the user, and/or an alertness of the user. Forexample, the visible light sensor 180 may control the lighting fixturesto increase lighting intensity if the user is at the desk 106 and theuser's eyes close for a predefined time. The visible light sensor 180may adjust the covering material 152 of the motorized window treatments150 to an open position and/or increase the lighting intensity oflighting fixtures if the user is in bed at a wake-up time. The visiblelight sensor 180 may adjust the lighting fixtures 172, 174, 176, 178based on default settings and/or based on a user preference.

The visible light sensor 180 may be configured to determine adepreciation in the light output of one or more of the lighting fixtures172, 174, 176, 178 in the room 102. The light output of the lightingfixtures 172, 174, 176, 178 may be depreciated as a result of age and/oruse of the lighting fixtures 172, 174, 176, 178. The depreciation of thelight output may result in the lighting fixtures 172, 174, 176, 178providing a lighting intensity that is less than preferred, a differentcolor temperature than is preferred, etc. The visible light sensor 180may be configured to control the lighting intensity of the lightingfixtures 172, 174, 176, 178 to compensate for the depreciation of thelighting fixtures 172, 174, 176, 178. For example, the visible lightsensor 180 may be configured to perform lumen maintenance of thelighting fixtures 172, 174, 176, 178.

The visible light sensor 180 may be configured to identify that thecolor temperatures provided by the lighting fixtures 172, 174, 176, 178has depreciated. The visible light sensor 180 may be configured tocontrol the lighting intensity of different colored LEDs in the lightingfixtures 172, 174, 176, 178 to compensate for the depreciation in colorof the lighting fixtures 172, 174, 176, 178. For example, the visiblelight sensor 180 may be configured to increase the intensity of the blueLED of the lighting fixtures 172, 174, 176, 178 to compensate for adepreciation in the blue LEDs, or increase the intensity of the red LEDof the lighting fixtures 172, 174, 176, 178 to compensate for adepreciation in the blue LEDs. The changing color temperature may becaptured in the images generated by the visible light sensor 180 and thevisible light sensor 180 may cease adjustment when the proper colortemperature is identified in the images.

Configuration and/or the operation of one or more of the devices withinthe load control system 100 (e.g., the visible light sensor 180) may beperformed using the mobile device 190 and/or another network device. Themobile device 190 may execute a graphical user interface (GUI)performance software for allowing a user to configure and/or operate theload control system 100. For example, the performance software may runas a PC application or a web interface (e.g., executed on the systemcontroller 110 or other remote computing device). The performancesoftware and/or the system controller 110 (e.g., via instructions fromthe configuration software) may generate a load control database thatdefines the operation of the load control system 100. For example, theload control database may include information regarding the controlsettings of different load control devices of the load control system(e.g., the lighting fixtures 172, 174, 176, 178, the plug-in loadcontrol device 140, the motorized window treatments 150, and/or thethermostat 160). The load control database may comprise informationregarding associations between the load control devices and the inputdevices (e.g., the remote control device 170, the visible light sensor180, etc.). The load control database may comprise information regardinghow the load control devices respond to inputs received from the inputdevices.

The load control database may define the control instructions for theload control devices in response to identification of different types ofinformation in each of the modes described herein. The controlinstructions may be defined by the user 192. Examples of configurationprocedures for load control systems are described in greater detail incommonly-assigned U.S. Pat. No. 7,391,297, issued Jun. 24, 2008,entitled HANDHELD PROGRAMMER FOR A LIGHTING CONTROL SYSTEM; U.S. PatentApplication Publication No. 2008/0092075, published Apr. 17, 2008,entitled METHOD OF BUILDING A DATABASE OF A LIGHTING CONTROL SYSTEM; andU.S. Patent Application Publication No. 2014/0265568, published Sep. 18,2014, entitled COMMISSIONING LOAD CONTROL SYSTEMS, the entire disclosureof which is hereby incorporated by reference.

The operation of the visible light sensor 180 may be programmed and/orconfigured using a network device, such as the mobile device 190. Thevisible light sensor 180 may comprise a communication circuit fortransmitting and receiving the RF signals 109 (for communicatingdirectly with the mobile device 190 (e.g., using a standard protocol,such as Wi-Fi or BLUETOOTH®). The visible light sensor 180 may also, oralternatively, communicate indirectly via the system controller 110.During the configuration procedure of the load control system 100, thevisible light sensor 180 may be configured to record an image of theroom 102 and transmit the image to the mobile device 190 (e.g., directlyto the mobile device 190 or indirectly via the system controller 110).The mobile device 190 may display the image on a visual display and theuser 192 may configure the operation of the visible light sensor 180 toset one or more configuration parameters (e.g., configuration data) ofthe visible light sensor 180. For example, the user 192 may indicateregions of interest on the image by tracing masked areas on the imagedisplayed on the visual display. The visible light sensor 180 may beconfigured to receive the modified images from the mobile device 190 andidentify the regions of interest and establish different masks and/orcontrol parameters depending upon the environmental characteristic to besensed (e.g., occupancy/vacancy conditions, light level inside of theroom 102, daylight level outside of the room 102, color temperature,etc.).

The mobile device 190 may transmit the configuration data to the visiblelight sensor 180. For example, the mobile device 190 may transmit to thevisible light sensor 180 the regions of interest defined by the user192, the preferred total light intensities (e.g., artificial lightand/or sunlight) defined by the user, and/or the preferred colortemperatures defined by the user 192. The mobile device 190 may also, oradditionally, transmit to the visible light sensor 180 the scenes to bedefined by the user. For example, the network device 192 may transmit tothe visible light sensor that a bedtime scene may include a warm colortemperature and/or the covering material 152 of the motorized windowtreatments 150 to be closed. The mobile device 190 may directly transmitthe configuration data to the visible light sensor 180 via the RFsignals 109 using the standard protocol and/or the mobile device 190 maytransmit the configuration data to the visible light sensor via one ormore other devices (e.g., the system controller 110). The visible lightsensor 180 may store the configuration data in memory, such that thevisible light sensor 180 may operate according to the providedconfiguration data.

As described herein, the visible light sensor 180 may be configured torecord an image of the room 102, which may capture one or more objectsand/or activities within the room that the user 192 may consider to beprivate or confidential. As a result, the visible light sensor 180 maybe configured to protect the privacy of the user 192, while using theimage data to configure and/or control load control devices within theroom 102.

The visible light sensor 180 may not be configured to transmit images,or be configured to prevent the transmission of images, during normaloperation. The visible light sensor 180 may be configured to only usethe images internally to sense the desired environmental characteristic(e.g., to detect occupancy or vacancy, to measure an ambient lightlevel, etc.). For example, the visible light sensor 180 may beconfigured to transmit (e.g., only transmit) an indication of thedetected state and/or measured environmental characteristic duringnormal operation (e.g., via the RF signals 108 using the proprietaryprotocol).

The visible light sensor 180 may be installed with special configurationsoftware for use during the configuration procedure (e.g., for use onlyduring the configuration procedure). The configuration procedure may beperformed prior to normal operation of the visible light sensor 180. Theconfiguration procedure may be performed dynamically during normaloperation to update the visible light sensor 180 (e.g., during normaloperation of the visible light sensor 180 or in response to the movementof an object). The configuration software may allow the visible lightsensor 180 to transmit a digital representation of an image recorded bythe camera to the mobile device 190 only during the configurationprocedure. The visible light sensor 180 may receive configuration datafrom the mobile device 190 (e.g., via the RF signals 109 using thestandard protocol) and may store the configuration data in memory. Thevisible light sensor 180 may have the configuration software installedduring manufacturing, such that the visible light sensor 180 is ready tobe configured when first powered after installation. In addition, thesystem controller 110 and/or the mobile device 190 may be configured totransmit the configuration software to the visible light sensor 180during the configuration procedure of the load control system 100.

The visible light sensor 180 may be configured to install normaloperation software in place of the configuration software after theconfiguration procedure is complete. The operation software may beconfigured for operating in the sensor modes described herein. Thenormal operation software may not allow the visible light sensor 180 totransmit images recorded by the camera to other devices. For example,during operation of the visible light sensor 180, the visible lightsensor 180 may be configured to transmit metadata of the image recordedby the camera. The visible light sensor 180 may have the normaloperation software stored in memory and may be configured to install thenormal operation software after the configuration procedure is complete.In addition, the system controller 110 and/or the mobile device 190 maybe configured to transmit the normal operation software to the visiblelight sensor 180 after the configuration procedure is complete.

Rather than installing special configuration software onto the visiblelight sensor 180 and then removing the special configuration softwarefrom the visible light sensor, a special configuration sensor (notshown) may be installed at the location of the visible light sensor 180(e.g., on or in place of the visible light sensor 180) or within apredefined distance of the location of the visible light sensor 180during configuration of the load control system 100. The configurationsensor may include the same camera and mechanical structure as thevisible light sensor 180. The configuration sensor may include a firstcommunication circuit for transmitting and receiving the RF signals 108using the proprietary protocol, for example, and a second communicationcircuit for transmitting and receiving the RF signals 109 using thestandard protocol, for example. During the configuration procedure ofthe load control system 100, the configuration sensor may be configuredto record an image of the space and transmit the image to the mobiledevice 190 (e.g., directly to the network device via the RF signals 109using the standard protocol). The mobile device 190 may display theimage on the visual display and a user may configure the operation ofthe visible light sensor 180. For example, the visible light sensor 180and the configuration sensor may be mounted to a base portion thatremains connected to the ceiling or wall, such that the configurationsensor may be mounted in the exact same location during configurationthat the visible light sensor is mounted during normal operation.

The configuration sensor may then be uninstalled and the visible lightsensor 180 may be installed in its place for use during normal operationof the load control system 100. The visible light sensor 180 for useduring normal operation may be incapable of transmitting images via theRF signals 109 using the standard protocol. The visible light sensor 180for use during normal operation may only comprise a communicationcircuit for transmitting and receiving the RF signals 108 using theproprietary protocol. After the visible light sensor 180 is installed,the mobile device 190 may transmit the configuration data to the systemcontroller 110 via the RF signals 109 (e.g., using the standardprotocol), and the system controller 110 may transmit the configurationdata to the visible light sensor via the RF signal 108 (e.g., using theproprietary protocol). The visible light sensor 180 may store theconfiguration data in memory of the sensor. During normal operation, thevisible light sensor 180 may transmit, for example, an indication of thesensed environmental characteristic during normal operation via the RFsignals 108 (e.g., using the proprietary protocol).

The visible light sensor 180 may comprise a removable configurationmodule for use during configuration of the visible light sensor 180. Thevisible light sensor 180 may use the removable configuration module(e.g., USB) during configuration of the visible light sensor 180 and theremovable configuration module may be removed for operation of thedevice for performing load control and execution of sensor modes. Thevisible light sensor 180 may comprise a first installed (e.g.,permanently-installed) communication circuit for transmitting andreceiving the RF signals 108 (e.g., using the proprietary protocol). Theremovable configuration module may comprise a second communicationcircuit for transmitting and receiving the RF signals 109 (e.g., usingthe standard protocol). When the configuration module is installed inthe visible light sensor 180 and the second communication circuit iselectrically coupled to the visible light sensor 180, the visible lightsensor 180 may record an image of the room 102 and transmit the image tothe mobile device 190 (e.g., directly to the network device via the RFsignals 109 using the standard protocol). The mobile device 190 maytransmit the configuration data to the visible light sensor 180 whilethe configuration module is still installed in the visible light sensor180, and the visible light sensor 180 may store the configuration datain memory. After the configuration of the visible light sensor 180(e.g., during operation of the visible light sensor 180), theconfiguration module may be removed from the visible light sensor 180.With the configuration module removed, the visible light sensor 180 maybe unable to transmit images via the RF signals 109 (e.g., using thestandard protocol).

The visible light sensor 180 may be configured to protect the privacy ofthe user 192 by disabling communication capabilities of the visiblelight sensor 180. For example, the visible light sensor 180 may beconfigured absent network capabilities, such as without a communicationcircuit for transmitting and/or receiving digital signals. The visiblelight sensor 180 configured absent network capabilities may be unable totransmit RF signals 108 (e.g., using the proprietary protocol), and thevisible light sensor 180 configured absent network capabilities may beunable to transmit RF signals 109 (e.g., using the standard protocol).The visible light sensor 180 configured, absent network capabilities,may be incapable of transmitting images of the room 102 to devicesexternal to the visible light sensor 180.

When the visible light sensor 180 is configured absent networkcapabilities, the visible light sensor 180 may be configured using acompletely automatic configuration procedure or using buttons on thevisible light sensor for manual configuration. For example, the visiblelight sensor 180 may include a visual display for displaying the imageand allow for configuration by user selections using the buttons orportions of the visual display. When the visible light sensor 180 isperforming normal operation absent network capabilities, the visiblelight sensor 180 may be wired directly to the load control devices forthe electrical loads being controlled.

The visible light sensor 180 may be configured to protect the privacy ofthe user 192 using an integrated circuit (IC) during normal operation.The visible light sensor 180 may record an image of a room 102 and theIC may modify the image. For example, the IC may decimate the image sothat one or more objects within the image are obfuscated and/orunrecognizable by the user 192. The IC may decimate the image by addingeffects to the image, such as by adding coarseness to the resolution ofthe image. The effects may be added to the image using layers and/or theeffects may be added directly to the image. The coarseness of theresolution may decimate the image so that the user 192 may not be ableto discern one or more objects within the image. For example, the IC maydecimate the image by converting the image to 16×16 pixels. The IC maybe programmed so that it is unmodifiable and/or hacked-proof. Forexample, the IC may be an application-specific integrated circuit (ASIC)that may be unmodifiable and/or hack-proof.

The visible light sensor 180 may be used to configure (e.g.,automatically configure) the load control system 100. The visible lightsensor 180 may be used to configure the load control system 100 bydetermining which of the control devices may be located within the room102. For example, during the configuration procedure of the load controlsystem 100, the visible light sensor 180 may instruct the load controldevices to perform an identifying function, such as flashing thelighting fixtures 172, 174, 176, 178 or change their color, raising orlowering the covering material 152 of the motorized window treatments150, flashing the lighting load in the floor lamp 142 through theplug-in load control device 140, blinking a visual indicator on one ormore devices, etc. The visible light sensor 180 may be configured todetect the identifying feature and determine the location of the objectsin the room 102.

The visible light sensor 180 may be configured to determine that one ormore control devices (e.g., the remote control device mounted to thewall, the temperature control device 160 mounted on the wall, thespeaker 146 mounted on the wall, etc.) are located in the room 102 fromthe visual appearance of the control device as shown in a recordedimage. After determining the control devices that are located in theroom 102, the visible light sensor 180 may receive unique identifiers(e.g., serial numbers) from these control devices and may generate(e.g., automatically generate) associations between the load controldevices and the input devices of the control devices located in theroom, or in regions of interest within the room in which the controldevices are located.

Devices may be identified to define regions of interest within the room.The control devices may be associated with the defined regions ofinterest and controlled according to their location within the room 102.For example, the computer monitor 166 may display an image or screencaptured by the images generated by the visible light sensor 180. Theimage or screen may be generated upon user actuation of a button on thekeyboard 168 or the mobile device 190. The visible light sensor 180 mayrecognize the image and define the computer monitor, or a predefinedarea around the computer monitor (e.g., the desk 106 or other predefinedarea) as the user's task area.

The lighting fixtures 172, 174, 176, 178 may each be turned on, turnedoff, or flashed to identify the impact of the lighting fixtures 172,174, 176, 178 on a region of interest. The visible light sensor 180 mayidentify a difference in lighting intensity (e.g., by a predefinedamount, such as moving the lighting intensity from one baseline intervalto another) within a region of interest when one of the lightingfixtures 172, 174, 176, 178 is turned on and/or off. When a lightingfixture 172, 174, 176, 178 is determined to impact the region ofinterest, the lighting fixture 172, 174, 176, 178 may be associated withthe region of interest for performing lighting control in the region ofinterest.

The visible light sensor 180 may be configured to identify one or moreobjects in the room 102, such as the door 105 and/or the window 104. Thevisible light sensor 180 may automatically identify the object (e.g.,the door 105 and/or the window 104) within the room 102 based onpredefined sizes and/or shapes of the object. The visible light sensor180 may automatically identify the object within the room 102 based onpredefined locations of the object. For example, the visible lightsensor may automatically identify the door 105 based on an object in theroom 102 being the size of a standard door and/or by the door 105 beingpositioned at a location within the room 102 at which a door may belocated. The visible light sensor 180 may be configured to mask one ormore objects within the room 102 during configuration and/or control ofthe load control system 100. For example, when an object (e.g., keyboard168) is located on a task surface (e.g., desk 106), a mask may beapplied to the object. An object located on a task surface may be a mug,a stapler, etc. Multiple masks may be applied at the same time. Forexample, when an object is located on a task surface (e.g., desk 106), afirst mask may be applied to the object on the task surface, whileanother mask is applied to an area surrounding (e.g., within apredefined distance of) the task surface.

Though functions may be described herein as being performed by thevisible light sensor, the visible light sensor may record the images andprovide the images to the system controller for performing imageanalysis, control procedures, and/or other functions described herein.

FIG. 3 is a simplified block diagram of an example visible light sensor300, which may be deployed as the visible light sensor 180 of the loadcontrol system 100 shown in FIG. 1. The visible light sensor 300 maycomprise a control circuit 310, for example, a microprocessor, aprogrammable logic device (PLD), a microcontroller, an applicationspecific integrated circuit (ASIC), a field-programmable gate array(FPGA), or any suitable processing device. The control circuit 310 maybe coupled to a memory 312 for storage of control parameters of thevisible light sensor 300. The memory 312 may operate as an externalintegrated circuit (IC) or as an internal circuit of the control circuit310. The memory 312 may have instructions stored thereon that, whenexecuted by the control circuit 310, enable the visible light sensor 300to perform the functions described herein.

The visible light sensor 300 may comprise a visible light sensingcircuit 320 having an image recording circuit, such as a camera 322. Thecamera 322 may be a removable configuration module, as described herein.The visible light sensor 300 may comprise an image processing circuit,such as an image processor 324. The image processor 324 may comprise adigital signal processor (DSP), a microprocessor, a programmable logicdevice (PLD), a microcontroller, an application specific integratedcircuit (ASIC), a field-programmable gate array (FPGA), or any suitableprocessing device. The camera 322 may be positioned towards a space inwhich one or more environmental characteristics are to be detected(e.g., into the room 102). The camera 322 may be configured to record animage which may be provided to the image processor 324. The imageprocessor 324 may be configured to process the image and provide to thecontrol circuit 310 one or more signals that are representative of thedetected environmental characteristics (e.g., an occurrence of movement,an amount of movement, a direction of movement, a velocity of movement,a counted number of users, a lighting intensity, a light color, anamount of direct sunlight penetration, etc.). For example, the one ormore signals provided to the control circuit 310 may be representativeof movement in the space and/or a measured light level in the space. Theimage processor 324 may provide an entire image to the control circuit310.

The visible light sensor 300 may comprise a first communication circuit330 configured to transmit and/or receive digital messages via a firstcommunication link using a first protocol. For example, the firstcommunication link may comprise a wireless communication link and thefirst communication circuit 330 may comprise an RF transceiver coupledto an antenna. In addition, or alternatively, the first communicationlink may comprise a wired digital communication link and the firstcommunication circuit 330 may comprise a wired communication circuit.The first protocol may comprise a proprietary protocol, such as, forexample, the ClearConnect protocol. The control circuit 310 may beconfigured to transmit and/or receive digital messages via the firstcommunication link during operation of the visible light sensor 300. Thecontrol circuit 310 may be configured to transmit an indication of thedetected environmental characteristic via the first communication linkduring operation of the visible light sensor 300. For example, thecontrol circuit 310 may be configured to transmit an indication of adetected state (e.g., an occupancy/vacancy condition) and/or a measuredenvironmental characteristic (e.g., a measured light level, color, etc.)via the first communication link during operation of the visible lightsensor 300.

The visible light sensor 300 may comprise a second communication circuit332 configured to transmit and/or receive digital messages via a secondcommunication link using a second protocol. For example, the secondcommunication link may comprise a wireless communication link and thesecond communication circuit 332 may comprise an RF transceiver coupledto an antenna. In addition, or alternatively, the second communicationlink may comprise a wired digital communication link and the secondcommunication circuit 332 may comprise a wired communication circuit.The second protocol may comprise a standard protocol, such as, forexample, the Wi-Fi protocol, the BLUETOOTH® protocol, the Zigbeeprotocol, etc. The control circuit 310 may be configured to transmit andreceive digital messages via the second communication link duringconfiguration of the visible light sensor 300. For example, the controlcircuit 310 may be configured to transmit an image recorded by thecamera 322 via the second communication link during configuration of thevisible light sensor 300.

The visible light sensor 300 may comprise a compass (e.g., an electroniccompass 334). The visible light sensor 300 may identify direction usingthe electronic compass 334. For example, the visible light sensor 300may identify the direction of one or more objects (e.g., window 104,door 105, etc.) using the electronic compass. The visible light sensor300 may comprise a power source 340 for producing a DC supply voltageV_(CC) for powering the control circuit 310, the memory 312, the imageprocessor 324, the camera, the first and second communication circuits330, 332, and other low-voltage circuitry of the visible light sensor300. The power source 340 may comprise a power supply configured toreceive an external supply voltage from an external power source (e.g.,an AC mains line voltage power source and/or an external DC powersupply). In addition, the power source 340 may comprise a battery forpowering the circuitry of the visible light sensor 300.

The visible light sensor 300 may further comprise a low-power occupancysensing circuit, such as a passive infrared (IR) detector circuit 350 orPIR detector circuit 350. The PIR detector circuit 350 may generate aPIR detect signal V_(PIR) (e.g., a low-power occupancy signal) that isrepresentative of an occupancy/vacancy condition in the room 102 inresponse to detected passive infrared energy in the room. The PIRdetector circuit 350 may consume less power than the visible lightsensing circuit 320. However, the visible light sensing circuit 320 maybe more accurate than the PIR detector circuit 350. For example, whenthe power source 340 is a battery, the control circuit 310 may beconfigured to disable the visible light sensing circuit 320 and use thePIR detector circuit 350 to detect occupancy conditions while conservingpower. The control circuit 310 may disable the light sensing circuit320, for example, when the room 102 is vacant. The control circuit 310may detect an occupancy condition in the room 102 in response to the PIRdetect signal V_(PIR) and may subsequently enable the visible lightsensing circuit 320 to detect a continued occupancy/vacancy condition.The control circuit 310 may enable the visible light sensing circuit 320immediately after detecting an occupancy condition in the room 102 inresponse to the PIR detect signal V_(PIR). The control circuit 310 mayalso keep the visible light sensing circuit 320 disabled after detectingan occupancy condition in the room 102 (e.g., in response to the PIRdetect signal V_(PIR)). The control circuit 310 may keep the visiblelight sensing circuit 320 disabled until the PIR detect signal V_(PIR)indicates that the room is vacant. The control circuit 310 may not makea determination that the room 102 is vacant until the visible lightsensing circuit 320 subsequently indicates that the room is vacant.

The visible light sensor 300 may be configured in a way that protectsthe privacy of the occupants of the space. For example, the controlcircuit 310 may execute special configuration software that allows thecontrol circuit 310 to transmit an image recorded by the camera 322 viathe second communication link only during configuration of the visiblelight sensor 300. The configuration software may be installed in thememory 312 during manufacturing, such that the visible light sensor 300is ready to be configured when first powered after installation. Inaddition, the control circuit 310 may be configured to receive theconfiguration software via the first or second communication links andstore the configuration software in the memory during configuration ofthe visible light sensor 300. The control circuit 310 may execute normaloperation software after configuration of the visible light sensor 300is complete. The normal operation software may be installed in thememory 312 or may be received via the first or second communicationlinks during configuration of the visible light sensor 300.

The second communication circuit 332 may be housed in a removableconfiguration module that may be installed in the visible light sensor320 and electrically connected to the control circuit 310 only duringconfiguration of the visible light sensor. When the configuration moduleis installed in the visible light sensor 300 and the secondcommunication circuit 332 is electrically coupled to the control circuit310, the control circuit may transmit an image recorded by the camera322 to via the second communication link. The control circuit 310 maysubsequently receive configuration data via the first or secondcommunication links and may store the configuration data in the memory312. The configuration module may then be removed from the visible lightsensor 300, such that the control circuit 310 is subsequently unable totransmit images via the second communication link.

In addition, the visible light sensor 300 that is installed in the spaceduring normal operation may not comprise the second communicationcircuit, such that the visible light sensor is never able to transmitimages via the second communication link. The visible light sensor 300may be configured using a special configuration sensor that may have anidentical structure as the visible light sensor 300 shown in FIG. 3 andmay include both a first communication circuit for communicating via thefirst communication link and a second communication circuit forcommunicating via the second communication link. The specialconfiguration sensor may be configured to record an image using thecamera and transmit the image via the second communication link. Thespecial configuration sensor may then be uninstalled and the visiblelight sensor 300 (that does not have the second communication link 332)may then be installed in its place for use during normal operation. Thecontrol circuit 310 of the visible light sensor 300 may receiveconfiguration data via the first communication link and may store theconfiguration data in the memory 312.

FIGS. 4A and 4B show sequence diagrams of example control procedures400, 400 a for controlling load control devices 406 using a visiblelight sensor 402 (such as visible light sensor 180) as described herein.As shown in FIG. 4A, the visible light sensor 402 may record an image,at 410. The visible light sensor 402 may record one or more images thatinclude one or more unmasked regions of interest. Based on the recordedimage, the visible light sensor 402 may, at 412, process the image todetect environmental characteristics of the one or more regions ofinterest (e.g., presence of sunlight on user's task area, a predefinedlighting intensity at users' task area, a color temperature in thespace, color temperature depreciation for one or more of light sources,an occupancy/vacancy condition, etc.). The environmental characteristicsmay be detected by comparing the recorded image with baseline image(s)of the space. The visible light sensor may transmit the detectedenvironmental characteristics to the system controller 404, at 414.

The system controller 404 may determine the command and/or the loadcontrol devices 406 to be controlled, at 416, based on the environmentalcharacteristics detected in the images. The command and/or the loadcontrol devices 406 to be controlled may be associated with the detectedenvironmental characteristics in memory. The command and/or the loadcontrol devices 406 to be controlled may be system defined and/or userdefined. A user preferences command may be provided to the visible lightsensor 402 for defining user preferences for controlling load controldevices. The user preferences command may derive from a network device.The system controller 404 may transmit the command, at 418, to the loadcontrol devices 406 in response to the characteristics detected from theimages.

FIG. 4A illustrates an example in which the system controller 402 isimplemented to determine a command and/or the load control devices 406to be controlled based on environmental characteristics detected in animage. The visible light sensor 402 itself may also, or alternatively,determine the command and/or the load control devices 406 to becontrolled based on the environmental characteristics detected from theimage. As shown in FIG. 4B, the visible light sensor 402 may record animage, at 420. Based on the recorded image, the visible light sensor 402may, at 422, process the image to detect environmental characteristics(e.g., presence of sunlight on user's task area, a predefined lightingintensity at users' task area, a color temperature in the space, colortemperature depreciation for one or more of light sources, anoccupancy/vacancy condition, etc.). The environmental characteristicsmay be detected by comparing the recorded image with baseline images ofthe space.

The visible light sensor 402 may determine the command and/or the loadcontrol devices 406 to be controlled, at 424, based on the environmentalcharacteristics detected in the images. The command and/or the loadcontrol devices 406 to be controlled may be associated with the detectedenvironmental characteristics in memory. The command and/or the loadcontrol devices 406 to be controlled may be system defined and/or userdefined. The visible light sensor 402 may transmit the command, at 426,to the system controller 404 for transmitting to the appropriate loadcontrol devices 406 for being controlled in response to the detectedenvironmental characteristics. The visible light sensor 402 may also, oralternatively transmit the command, at 428, to the load control devices406 in response to the characteristics detected from the images. If thevisible light sensor 402 transmits the command, at 428, directly to theload control devices 406, the system controller 404 may hear the commandand maintain the status of the load control devices 406 in memory. Thevisible light sensor 402 may continue to monitor regions of interest byrecording images and the load control devices may be controlled based onthe environmental characteristics in the space.

FIG. 5 shows a flowchart of an example sensor event procedure 500 thatmay be executed by a sensor (e.g., the visible light sensor 180, 300).The sensor event procedure 500 may be executed by a control circuit ofthe sensor (e.g., the control circuit 310) to step through sensor eventsto detect a plurality of environmental characteristics of a space (e.g.,the room 102 or the room 200). For example, the sensor event procedure500 may begin at step 510 during normal operation of the sensor. At step512, the control circuit may determine the next sensor event that may bestored in memory. For example, the first time that the control circuitexecutes step 512, the control circuit may retrieve the first sensorevent from memory. The control circuit may then retrieve an image from acamera and/or an image processor of the sensor (e.g., the camera 322and/or the image processor 324) at step 514. For example, the controlcircuit may retrieve a raw image (e.g., a frame acquisition from thecamera 322) or a preprocessed image (e.g., a background-subtractedimage).

At step 516, the control circuit may determine an algorithm to use toprocess the image to detect the environmental characteristic of thepresent sensor event. At step 518, the control circuit may determinecontrol parameters to use when executing the algorithm for the presentsensor event. At step 520, the control circuit may apply a mask(s)(e.g., that may be stored in memory for the present sensor event) to theimage (e.g., that may be retrieved at step 514) to focus on one or moreregions of interest in the image. The control circuit may then processthe region of interest of the image using the determined algorithm andcontrol parameters of the present sensor event at step 522 and transmitthe result (e.g., via RF signals 108 using the first communicationcircuit 330) at step 524. If the control circuit should continue normaloperation at step 526, the sensor event procedure 500 may loop around toexecute the next sensor event at steps 512-524. If the control circuitshould cease normal operation at step 526 (e.g., in response to a userinput to cease normal operation or other interrupt to normal operation),the sensor event procedure 500 may exit.

FIG. 6 shows a flowchart of an example occupancy/vacancy detectionprocedure 610 that may be executed by a sensor (e.g., the visible lightsensor 180, 300) for detecting occupancy and/or vacancy sensor events orconditions in a space. The occupancy/vacancy detection procedure 610 maybe performed by a single device, or distributed across multiple devices.The occupancy/vacancy detection procedure 610 may be executed by acontrol circuit of the sensor (e.g., the control circuit 310) during anoccupancy/vacancy sensor mode of the sensor for detectingoccupancy/vacancy sensor events. As shown in FIG. 6, theoccupancy/vacancy detection procedure 610 may begin at 610. At 612, animage may be retrieved. The image may be a pre-recorded image retrievedfrom memory, or recorded and retrieved from an image processor orcamera. At 614, a background of the unmasked area may be established.The background may be a static image of the space. The background may bepredefined or established while the sensor is taking images of the spaceover a period of time. For example, the background may be establishedover a period of time using a Gaussian mixture model. The background mayinclude objects within the image and a static location of the objectsover the period of time. At 616, a mask may be applied to the image,such that the masked portions of the image may not be processed.

A sensitivity level may be determined at 618 for processing the image.The sensitivity level may be adjusted to prevent false triggers foroccupancy based on smaller movements within the images. Highersensitivity levels may trigger an occupancy condition when smallermovements are identified in the image, while lower sensitivity levelsmay trigger an occupancy condition when greater movements are identifiedin the image. The sensitivity levels may be user-configured. Thesensitivity levels may change based on the region of interest. Forexample, a lower sensitivity level may be set when the region ofinterest is an entire room, whereas a higher sensitivity level may beset when the region of interest is on an area of a user's desk (e.g.,keyboard, etc.) or other task area within the room.

An image processing threshold may be set based on the determinedsensitivity at 620. The image processing threshold may be used toconvert to a binary image. To create a binary image from grayscale, thegrayscale level (e.g., between white and black) of each pixel may becompared to the image processing threshold to set the grayscale pixelsas black or white pixels in the binary image based on the side of thecolor image processing threshold that the pixels reside. In an example,the image processing threshold may be set to 50%, or a value of 128, anda pixel color value above 128 may be set to white, while a pixel colorvalue at or below 128 may be set to black.

At 622, a difference between the background and the present frame may bedetermined. The difference may indicate a change in a location of one ormore objects within the space. The differences between the backgroundand the present frame may be converted, at 624, to a binary image basedon the processing threshold.

The binary image may include one or more binary large objects (BLOBs).At 626, the control circuit may apply morphological operators to theBLOBs in the binary image. The morphological operators may include oneor more of a close operation and an open operation. The close operationmay be a dilation operation followed by an erosion operation, while theopen operation may be an erosion operation followed by a dilationoperation. The close operation may fill in small gaps in the BLOBs. Theopen operation may remove stray, small BLOBs from the binary image.Together, the close and open operations may improve ragged edges, fillin small gaps and remove small features of the BLOBs. At 628, thecontrol circuit may detect connected regions in the BLOBs to defineregions that are individual BLOBs (e.g., distinct BLOBs) versus a regionthat is a single BLOB, for example, using a connected-component labelingalgorithm. The connected components labeling algorithm will identify andlabel the separate BLOBs.

At 630, a determination may be made as to whether the size of one ormore of the BLOBs is greater than or equal to a predefined sizethreshold (e.g., a fixed detection threshold), which may indicatemovement of an occupant in the space and thus an occupancy sensor event.If the size of one or more of the BLOBs is greater than or equal to thepredefined size threshold (e.g., if a determination of occupancy hasbeen made), a determination may be made as to whether the visible lightsensor is operating in an occupied state at 632. If the visible lightsensor is not currently in the occupied state, the visible light sensormay change to operation in the occupied state at 634. After the changeto the occupied state at 634, the visible light sensor may start avacancy timer at 636 and transmit an occupied message at 638 thatindicates that the space is occupied. If no more determinations of theoccupancy condition are made at 630 (e.g., when the occupancy/vacancydetection procedure 610 is subsequently executed), the vacancy timer mayrun for a predetermined period of time prior to switching the visiblelight sensor back to the vacancy state. If the visible light sensor iscurrently operating in an occupied state, the visible light sensor mayreset a vacancy timer at 640, and transmit the occupied digital messageat 642 to indicate continued movement of the object in the space andthat the space is still occupied. If the size of one or more of theBLOBs is less the predefined size threshold (e.g., if a determination ofvacancy has been made) at 630, the occupancy/vacancy detection procedure610 may end (e.g., without resetting or stopping the vacancy timerand/or without transmitted an occupied or a vacant message).

FIG. 7 is a flowchart of an example vacancy timer procedure 700 executedby a visible light sensor, such as the visible light sensor 180 shown inFIG. 1 and/or the visible light sensor 300 shown in FIG. 3. The controlprocedure 700 may operate during an occupancy/vacancy sensor mode of thevisible light sensor 180, 300.

The vacancy timer procedure 700 may be executed periodically by thecontrol circuit 310 of the visible light sensor 300 control circuit 310when the vacancy timer expires at step 710. The control circuit 310 maychange to the vacant state at step 712 and transmit a vacant message(e.g., via the first communication link using the proprietary protocol)at step 714. The vacancy timer procedure 700 may exit. When the visiblelight sensor is operating with an infrared (IR) sensor for detectingoccupancy/vacancy, the control circuit 310 may disable the visible lightsensing circuit 320 until the next occupancy is detected by the IRsensor. Though the vacancy timer procedure 700 is describes as beingoperated by the visible light sensor, the vacancy timer procedure 700,or portions thereof, may be operated by the system controller.

FIG. 8 shows a flowchart of an example baseline configuration procedure800 for generating and storing baselines (e.g., one or more baselineimages and/or one or more baseline intensity levels) a of a space (e.g.,the room 102). The baseline configuration procedure 800 may be executedby a sensor, such as the visible light sensor 180, 300. The baselineconfiguration procedure 800 may be executed by a control circuit of thesensor (e.g., the control circuit 310) during a configuration procedurefor configuring operation of the sensor during a daylighting sensormode. The baseline configuration procedure may be executed when daylightand/or ambient light is minimized in the space, for example, at night,when the covering material of the motorized window treatments in thespace are in a fully closed position, and/or at another time whendaylight and/or ambient light are minimized in the space.

The baseline configuration procedure 800 may begin at 810. At 812, thesensor may set the lighting intensity level to a starting lightingintensity level. The starting lighting intensity level may be a minimumlighting intensity (e.g., approximately 1%), a maximum lightingintensity (e.g., approximately 100%), and/or a lighting intensitybetween the minimum intensity and the maximum intensity. The sensor mayset the lighting intensity to a starting intensity level by adjusting(e.g., via control instructions) one or more lighting control devices(e.g., lighting fixtures 172, 174, 176, 178).

At step 814, the sensor may record an image. For example, the sensor mayrecord one or more images of one or more regions of interest within thespace (e.g., the room 102). The sensor may store the recorded image asthe one or more baseline image at 816. The sensor may calculate alighting intensity level that may represent, for example, an averagelight level in the baseline image may be stored at 816 (e.g. as shown inFIG. 10A), and may set the calculated lighting intensity level as abaseline lighting intensity level.

The sensor may record the control setting of one or more control devices(e.g., lighting fixtures 172, 174, 176, 178; motorized window treatments150, etc.) that were used when the baseline image was recorded and/orthe baseline lighting intensity was determined, and, at step 818,associate the control settings with the baseline lighting intensityimage. For example, the sensor may associate lighting fixtures having adimming level of 25% with the baseline lighting intensity in therecorded baseline image.

A similar process may be performed at different lighting intensitylevels to store a baseline image and associated control settings for thedifferent lighting intensity levels. As such, the sensor may begin at astarting intensity level (e.g., 1% or 100%) and step wise (e.g.,incrementally) adjust the lighting intensity level by a predefinedamount (e.g., 1%, 10%, 25%, 33%, etc.,) to record and store thebaselines and the corresponding control settings. The sensor maydetermine, at 820, if the lighting intensity level is the endinglighting intensity level. For example, if the lighting intensity levelis a 100% intensity level and the ending level is a maximum intensity(e.g., approximately 100%), the lighting intensity level and the endinglighting intensity level may be the same. If the lighting intensitylevel and the ending lighting intensity level are the same, theprocedure 800 may end. If the lighting intensity level and the endinglighting intensity level are different, the sensor may step up (e.g.,incrementally adjust) the lighting intensity level (e.g., the lightingintensity level within room 102) by a predefined amount at 822 andproceed to 812. After adjusting the lighting intensity level, theprocedure 800 may move to 814 and record an image at the adjustedlighting intensity level for storing as a baseline image with anassociated control setting of the light source.

FIG. 9A shows a flowchart of an example procedure 900 for determiningthe impact of light emitted by lighting fixtures on sub-areas of aspace. The procedure 900 may be executed by a sensor, such as thevisible light sensor 180, 300. The procedure 900 may be executed by acontrol circuit of the sensor (e.g., the control circuit 310) at a timewhen daylight and/or ambient light in the space is minimized, so thatthe contribution of the artificial light emitted by the lightingfixtures can be determined without being affected by the contribution ofdaylight and/or ambient light. The procedure 900 may be executed atnighttime and/or when the covering material of the motorized windowtreatments in the space are in a closed state to prevent entry ofdaylight light into the space. The images that are stored during theprocedure 900 may be used during operation of a daylighting sensor modeof the sensor.

The procedure 900 may begin at 910. At 912, a next minimally-controlledzone may be turned on. A minimally-controlled zone may include one ormore lighting loads (e.g., of lighting fixtures 172, 174, 176, 178) thatmay be independently controlled. For example, where each lightingfixture 172, 174, 176, 178 may be independently controlled, eachminimally-controlled zone may include a single lighting load, whichmeans that the next lighting fixture may be turned on at 912. Eachminimally-controlled zone may have a single lighting load or multiplelighting loads. When the minimally-controlled zones have multiplelighting loads, the zones may have the same number or different numbersof lighting loads. At 914, the sensor may record an image of the space.A daylighting mask may be applied to the image at 916. The daylightingmask may be applied to areas outside of a task area or other region ofinterest, or to areas unaffected by the light emitted by the daylightingzone.

A subtraction process may be performed to the image, at 918, to removebright and/or dark spots within the image. The bright and/or dark spotsmay represent locations in the image (e.g., on the task area or otherregion of interest) at which an object may be located. The bright and/ordark spots may represent locations in the image at which reflected lightmay be shining or a shadow may be located. The objects in the image maynot reflect light similarly to the rest of the space (e.g., the taskarea or other region of interest). The subtraction process may beperformed on portions of the image that are above or below a predefinedsize (e.g., number of pixels) and/or contrast threshold. The subtractionprocess may be performed by subtracting portions of the image above apredefined brightness threshold and/or below a predefined darknessthreshold. Rather than subtracting the portions of the image, a mask maybe applied to portions of the image that are above or below thepredefined size (e.g., number of pixels) and/or contrast threshold

The excluded bright and/or dark spots may be backfilled, at 920, tosimulate the light emitted by the one or more lighting loads in the zonethat are on, if the objects creating the bright and/or dark spots werenot present in the image. The removed spots may be backfilled at 920with an average lighting intensity value of adjacent pixels to thespots. The nighttime image may be stored for the zone at 922 to identifythe contribution of the artificial light emitted by the one or morelighting loads in the zone without being affected by the contribution ofdaylight and/or ambient light.

At 924, a determination may be made whether additional zones are yet tobe processed or updated. If additional zones are to be processed orupdated, the procedure 900 may return to 912 and the nextminimally-controlled zone may be turned on for recording and processingan image when the one or more lighting loads of the zone are turned on.If no additional zones are to be processed or updated, the light impactof each fixture may be determined for each sub-area of the space (e.g.,the task area or other region of interest). A sub-area may be comprisedof a group of pixels identified within the image (e.g., within theunmasked area of the image).

The light impact of each lighting fixture in a sub-area (e.g., of thetask area or other region of interest) may be determined, at 926, tounderstand the fixtures that have the greatest impact on the lightingintensity in each sub-area when turned on. The light impact of eachfixture in a sub-area (e.g., of the task area or other region ofinterest) may be determined, at 926, using recorded images of eachlighting fixture being turned on in the space, or one or more zones inthe space, and analyzing the sub-area of the image to identify thecontribution of the lighting intensity reflected by the light fixture inthe sub-area. If there are more sub-areas (e.g., of the task area orother region of interest) for which the light impact per fixture is tobe determined at 928, the procedure 900 may return to 926 to determinethe light impact of each fixture in the next sub-area. When the lightimpact of each sub-area in the space (e.g., the task area or otherregion of interest) has been determined, the fixtures may be ranked byimpact per sub-area at 930. The ranking may be used to understand thefixtures to be controlled first to have the greatest impact on thelighting level of sub-areas in the space (e.g., on the task area orother region of interest) during a daylighting sensor mode, as shown inFIGS. 12a and 12B, for example. The lighting fixtures having thegreatest impact on the lighting level of a sub-area may be controlledfirst to more quickly reach a desired lighting level or uniform lightinglevel in the space (e.g., on the task area or other region of interest.

FIGS. 9B-9E show a set of example nighttime images 950 a, 950 b, 950 c,950 d of a conference room. The nighttime images 950 a, 950 b, 950 c,950 d have a mask 956, such as a daylighting mask, applied to the imagesmask off the areas of the image outside of a user task surface 952, suchas a desk, so that analysis can be performed on the unmasked user tasksurface 952.

Each image 950 a, 950 b, 950 c, 950 d illustrates the lighting levelcontribution of a different lighting fixture without the other lights inthe conference room being on, or daylight being present in the room. Thedark portions 954 a, 954 b, 954 c, 954 d represent the portions of therespective images 950 a, 950 b, 950 c, 950 d that have a higher lightingintensity, which is caused by the lighting fixture being on. Thelighting fixtures in each image 950 a, 950 b, 950 c, 950 d may be turnedon at a full intensity (e.g., 100%). The nighttime images 950 a, 950 b,950 c, 950 d may be analyzed to identify the contribution of theartificial light emitted by each lighting fixture without being affectedby the contribution of daylight. If the user task surface 952 has foursub-areas (e.g., one in each quadrant of the user task surface 952), thenighttime images 950 a, 950 b, 950 c, 950 d may be used to determine theimpact of each lighting fixture on the respective sub-area.

FIG. 10A shows a flowchart of an example procedure 1000 for measuringand controlling a lighting level or daylight lighting level on a taskarea or other region of interest in a space (e.g. the room 102). Theprocedure 1000 may be executed by a visible light sensor, such as thevisible light sensor 180. The procedure 1000 may be executed by acontrol circuit of the sensor (e.g., the control circuit 310) during adaylighting sensor mode of the visible light sensor for controlling thelighting loads for daylighting in the space.

The procedure 1000 may begin at 1010. At 1012, an image of the task areaor other region of interest may be retrieved. A previously capturedimage may be retrieved from memory or the image may be retrieved bycapturing the image with a camera. A daylighting mask may be applied tothe image at 1014. The daylighting mask may be applied to areas outsideof the task area or other region of interest, and/or to areas unaffectedby the light emitted by the daylighting zone.

A subtraction process may be performed to the image, at 1016, to removebright and/or dark spots within the image. The bright and/or dark spotsmay represent locations in the image (e.g., on the task area or otherregion of interest) at which an object may be located. The objects inthe image may not reflect light similarly to the rest of the space(e.g., the task area or other region of interest). The subtractionprocess may be performed by subtracting portions of the image that areabove a predefined brightness threshold and/or below a predefineddarkness threshold. The removed bright and/or dark spots may bebackfilled, at 1018, to simulate the light emitted by the one or morelighting loads in the zone that are on, if the objects creating thebright and/or dark spots were not present in the image. The removedspots may be backfilled at 1018 with an average lighting intensity valueof adjacent pixels to the spots.

FIGS. 10B-10D shows an image 1050 a for illustrating the subtraction andbackfill process. The image 1050 a may have a mask 1056, such as adaylighting mask, applied to the image to mask off areas of the imageoutside of a user task surface 1052, such as a desk, so that analysiscan be performed on the unmasked user task surface 1052. As shown inFIGS. 10B-10D, artificial light may be shining on the user task surface1052 from lighting fixtures above the user task surface and daylight maybe shining on the user task surface through windows 1054. The darkerportions of the user task surface 1052 may represent areas of higherlighting level, which are on the side of the task surface 1052 that iscloser to the windows 1054 as shown in FIG. 10B-10D.

As shown in FIG. 10B, the task surface 1052 may include bright spots,such as bright spot 1060 represented with a black spot, and dark spots,such as dark spot 1058 represented as a white spot. FIG. 10C shows theresult of the subtraction process (e.g., illustrated in step 1016 ofFIG. 10A) being applied to the bright spot 1060 and the dark spot 1058shown in FIG. 10B. As shown in FIG. 10C, the subtraction process mayremove portions 1064 and 1062 that previously included the bright spot1060 and the dark spot 1058, respectively. The removed portions 1062 and1064 may be the portions that were above a predefined brightnessthreshold and below a predefined darkness threshold. FIG. 10D shows theimage 1050 a after the backfill process is performed (e.g., asillustrated in step 1018 of FIG. 10A).

Referring again to FIG. 10A, at 1020, a baseline removal process may beperformed. If the baseline removal process is performed, at 1020, theprocedure 1000 may calculate daylight lighting level. If the baselineremoval process is not performed, at 1020, the procedure may calculatethe total lighting level (e.g., daylight and artificial light) in thespace. The baseline removal process may be performed as a function ofbaseline images, which may be images taken at nighttime or another timeduring which the artificial light in the space is unaffected by daylightor other ambient light, and the currently retrieved image to remove theportion of the artificial lighting intensity that is contributed by thelighting loads indicated in the baseline images. The baseline images mayreflect a baseline lighting intensity in the space when one or more ofthe lighting loads are turned on to their full potential (e.g., 100%intensity) or to another control setting at which the lighting loads arecurrently being controlled. The removal of the baseline lightingintensity from the current image may indicate the amount of naturallight or ambient light being added to the space.

FIGS. 10E-10G illustrate the baseline process (e.g., as illustrated instep 1020 of FIG. 10A) being applied to the image 1050 a. As shown inFIG. 10E the image 1050 a may include a task surface with light and darkspots removed (e.g., similar to FIG. 10D). FIG. 10F shows a baselineimage 1050 b, which may represent the contribution of the light emittedby the lighting fixtures to the task surface 1052 when the lightingfixtures are on at their present intensities. The baseline image 1050,may be recorded with the lighting fixtures on to the present intensities(e.g., during the procedure 800 of FIG. 8), or may be generated as thecombination of the nighttime images shown in FIGS. 9B-9E that arerecorded at maximum intensity (e.g., during the procedure 900 of FIG.9A) and then scaled to the present intensities of the lighting fixtures.In the example baseline image 1050 b shown in FIG. 10F, the lightingfixtures may be on at the same intensity level resulting in a constantlighting level by the lighting fixtures across the task surface 1052(e.g., what the image 1050 a would look like if taken at night with thelights at the present levels). FIG. 10G shows the image 1050 a after thebaseline removal process (e.g., as illustrated in step 1020 of FIG.10A). The image 1050 a shown in FIG. 10G is the result of the baselineimage 1050 b shown in FIG. 10F being subtracted from image 1050 a inFIG. 10E. The image 1050 a shown in FIG. 10G may illustrate the daylightcontribution 1058 on the task surface 1052, e.g., without the artificiallight contribution.

Referring again to FIG. 10A, an average lighting level metric may becalculated over the unmasked area at 1022. The average light levelmetric may indicate the contribution to the total lighting intensity, ortotal daylight contribution, on the task area or other region ofinterest resulting from the current control setting of the lightingcontrol devices and/or by any daylight or other ambient light. Thecalculated light level metric may be transmitted at 1024. The lightlevel metric may be transmitted from the visible light sensor to thesystem controller and/or one or more load control devices, when thelight level metric is calculated at the visible light sensor. The lightlevel metric may be transmitted from the system controller to one ormore load control devices, when the light level metric is calculated atthe system controller. The light level metric may be used to increase ordecrease the lighting intensity level of one or more lighting loads inthe lighting fixtures to control a total lighting level on a task areaor other region of interest. The lighting intensity level may beincreased or decreased to reach a target total lighting level on thetask area or other region of interest.

When the daylight lighting level is calculated (e.g., using the baselineremoval process at step 1020), the visible light sensor may determinehow much lighting intensity to produce from the lighting fixtures toachieve the target lighting level after the daylight contribution issubtracted. When the total lighting level is calculated (e.g., withoutthe baseline removal process at step 1020), the visible light sensor maydetermine how much lighting intensity to produce from the lightingfixtures to achieve the target lighting level.

FIG. 11 shows a flowchart of another example procedure 1100 formeasuring and controlling a total lighting level (e.g. illuminance orluminance) on a task area or other region of interest in a space (e.g.room 102). The procedure 1100 may be executed by a visible light sensor,such as the visible light sensor 180. The procedure 1100 may be executedby a control circuit of the sensor (e.g., the control circuit 310)during a daylighting sensor mode of the sensor for controlling thelighting loads for daylighting in the space.

The procedure 1100 may begin at 1110. At 1112, an image of the task areaor other region of interest may be retrieved. A previously capturedimage may be retrieved from memory or the image may be retrieved bycapturing the image with a camera. A daylighting mask may be applied tothe image at 1014. The daylighting mask may be applied to mask of (e.g.,exclude) areas outside of the task area or other region of interest, orto areas unaffected by the light emitted by the daylighting zone.

A subtraction process may be performed to the image, at 1116, to removebright and/or dark spots within the image. The bright and/or dark spotsmay represent locations in the image (e.g., on the task area or otherregion of interest) at which an object may be located. The objects inthe image may not reflect light similarly to the rest of the space(e.g., the task area or other region of interest). The bright and/ordark spots may represent locations in the image at which reflected lightmay be shining or a shadow may be located. The subtraction process maybe performed by subtracting portions of the image that are above apredefined brightness threshold and/or below a predefined darknessthreshold. The removed bright and/or dark spots may be backfilled, at1118, to simulate the light emitted by the one or more lighting loads inthe zone that are on, if the objects creating the bright and/or darkspots were not present in the image. The removed spots may be backfilledat 1118 with an average lighting intensity value of adjacent pixels tothe spots.

An average light level metric may be calculated over the unmasked areaat 1122. The average lighting level metric may indicate the contributionto the total lighting intensity at the task area or other region ofinterest (e.g., luminance or illuminance) resulting from the currentcontrol setting of the lighting control devices and/or any daylight orother ambient light. A determination may be made, at 1124, as to whetherthe calculated light level metric is within predefined upper and/orlower limits. The predefined limits may be preferred total lightinglevels for the task area or other region of interest. The preferredlimits may depend on user preferences and/or the task that is beingperformed at the task area or other region of interest. For example, anengineer may prefer a lighting intensity of twenty foot candles, while ageneral task worker may prefer a lighting intensity of fifty footcandles. The visible light sensor may define the limits for thecalculated light level metric after identifying one or more of the userswithin the space. For example, the visible light sensor may identify auser from an image of the user, from a unique identifier of a networkdevice used by the user within the room, audio identification, loginidentification, etc.

If the calculated lighting level metric is within the predefined limits,the procedure 1100 may end. If the calculated light level metric isoutside of the predefined light predefined limits, the sensor maytransmit one or more commands to adjust the intensity level of the lightsources (e.g., lighting fixtures 172, 174, 176, 178), at 1126. Thesensor may continue to monitor the space during the adjustment and theprocess 1100 may return to 1112. The sensor may stop adjusting thelighting intensity level of the light sources when the calculated lightlevel metric is identified as being within the predefined limits. Thesensor may also, or alternatively, identify whether moving the coveringmaterial of the motorized window treatments may increase or decrease thetotal lighting level of the task area or other region of interestwithout creating undesired daylight glare. If the covering material maybe increased or decreased to change the total lighting level of the taskarea or other region of interest without creating undesired daylightglare, a command may be transmitted to the motorized window treatmentsto affect the total lighting level.

FIGS. 12A and 12B show a flowchart of an example procedure 1200 forcontrolling the lighting fixtures to provide a uniform predefined lightprofile on a task area or other region of interest. The procedure 1200may be executed by a visible light sensor, such as the visible lightsensor 180. The procedure 1200 may be executed by a control circuit ofthe sensor (e.g., control circuit 310) during a daylighting sensor modeof the sensor for controlling lighting loads for daylighting in thespace. The predefined light profile may be a uniform light profile toachieve uniform light levels across the task area, or a gradient profileto achieve varying light levels across the task area (e.g., from a firsttarget light level, such as a measured daylight level, to a secondtarget light level, such as a desired interior light level).

The procedure 1200 may begin at 1210. At 1212, an image of the task areaor other region of interest may be retrieved. A previously capturedimage may be retrieved from memory or the image may be retrieved bycapturing the image with a camera. A map (e.g., a target map) of thetask area or other region of interest may be established, at 1214. Themap may define the target lighting levels for each sub-area of anunmasked portion of the image, e.g., the task area or other region ofinterest. For example, the target map may define a uniform lightprofile, or a gradient profile. The map may be defined as a target lightprofile for the region of interest, which indicates a target lightinglevel for different sub-areas within the region of interest. Thesub-areas may define areas (e.g., groups of one or more pixels) of theunmarked portion of the image that may be analyzed to achieve the targetlight levels defined by the target map. For example, if the target mapdefines a uniform light profile, the sub-areas may each have the sametarget light level. If the target map defines a gradient profile, thesub-areas may have different target lighting levels. A sub-area may beas small as a pixel, such that processing may be performed on apixel-by-pixel basis. Sub-areas may also be groups of multiple pixels tosave on processing power. Each sub-area may include the same number ofpixels or pixels within a predefined range of one another. Each sub-areamay be distinct without overlapping with another sub-area.

The sub-areas may be established during a configuration procedure (e.g.,using a network device, such as the mobile device 190). For example, theimage may be displayed to the user on the network device and the usermay draw on the image of the task surface, or other region of interest,during configuration of the sensor to establish each sub-area of thetask surface. The sub-areas may be established in a similar manner asthe regions of interest. The user may also, or alternatively, select anumber of sub-areas in a region of interest on the network device, andthe network device and/or sensor may automatically divide the region ofinterest into the number of sub-areas. The region of interest may beautomatically divided by detecting sub-areas having the closest lightlevels.

If the target lighting level is to be uniform across the region ofinterest, the user may enter a desired target lighting level to beapplied across the region of interest at the network device, which maybe communicated to the sensor. The user may also, or alternatively,specify a gradient across the region of interest and enter the lightinglevel at each end of the gradient (e.g., a measured daylight lightinglevel near a window and a desired lighting level at the interior of thespace). When the user selects the end-point lighting level levels forthe gradient, the visible light sensor may receive the endpoint lightinglevels and automatically determine the target lighting levels for eachsub area based on the entered end-point lighting levels. The sub-areasmay be automatically divided by detecting sub-areas having the closestlighting levels. The target lighting levels of the gradient may beentered manually on the mobile device and sent to the visible lightsensor. For example, a lighting designer may want the ability to definesub-areas of a region of interest and/or enter a target lighting levelfor each sub-area.

The sub-areas and/or the target lighting levels may be established(e.g., originally or updated) by the network device and/or the sensorlearning the sub-areas and/or the target lighting levels. The networkdevice and/or the sensor may identify common regions of interest and/orsub-areas of the region of interest by analyzing images of the occupantin the space. The network device and/or the sensor may identify commonlighting levels at the task surfaces, or sub-areas of the task surfaces,and set the common lighting levels automatically.

At 1216, a measured lighting level may be determined in each sub-area ofthe task area or other region of interest within the image. The measuredlighting level may be determined by a process similar to the procedure1000 shown in FIG. 10A. To calculate the total light level in eachsub-area at 1216, step 1020 of the procedure 1000 of FIG. 10A may beeliminated. Alternatively, the daylight contribution in each sub-areamay be determined at 1216, in which case step 1020 of the procedure 1000of FIG. 10A may be executed. A difference between the measured lightlevel in each sub-area and a target light level of each sub-area may bedetermined at 1218. The difference may be calculated by subtracting thetarget lighting level from the measured lighting level in each sub-area.An over-illuminated sub-area may be indicated by positive values aftercalculating the difference. An under illuminated sub-areas may beindicated by negative values after calculating the difference. Thesub-areas having the target lighting levels may be indicated by a valueof zero after calculating the difference. At 1222, the sub-areas mayeach be ranked based on greatest difference between the measuredlighting level and the target lighting level. The ranking may identifythe sub-areas having the greatest deficit in total lighting levels fromthe target lighting level (e.g., under-illuminated sub areas). At 1224,a lighting fixture may be selected that has the greatest influence onthe light level of the sub-area (e.g., from the rank of the fixturescalculated in FIG. 9A) that is the furthest from the target. Forexample, the lighting fixture may be selected that has the greatestinfluence on the sub-area of the task area that is the darkest.

At 1226, the visible light sensor may estimate the effect of the mostinfluential fixtures, selected at 1224, on the sub-area bymathematically adjusting the intensity level on to obtain the estimatedaffect. The sensor may mathematically increase the intensity level ofthe most influential fixture, selected at 1224, on the sub-area by anincrement (e.g., 1%, 5%, 10%, etc.). This mathematical increase may beperformed by multiplying the baseline image of the selected fixture bythe dimming level of the fixture. At 1226, the sensor may calculate theestimated lighting level of the sub-area after the intensity level ofthe selected fixture has been increased by an increment to determinewhether the target lighting level has been reached in the presentsub-area (e.g., the sub-area having the greatest deficit below thetarget light level). If the calculated lighting level of in the presentsub-area is determined, at 1228, not to be greater than or equal to thetarget lighting level, a determination may be made, at 1230, as towhether the currently selected fixture is at a full intensity. If thecurrently selected fixture is not at full intensity, the visible lightsensor may continue to mathematically increase the intensity level ofthe currently selected fixture and calculate the estimated lightinglevel of the sub-area after the increase in the intensity level untilthe target lighting level is reached or the currently selected fixtureis at a full intensity.

If the sensor determines that the currently selected fixture is at afull lighting intensity, the lighting fixture that has the next greatestinfluence on the lighting level of the sub-area (e.g., from the rank ofthe fixtures calculated in FIG. 9A) may be selected, at 1232. Thelighting fixture that has the next greatest influence on the light levelof the present sub-area may be determined from the rank of the fixturesdetermined in FIG. 9A. The intensity level of the next most influentialfixture on the sub-area may be mathematically increased by an increment(e.g., 1%, 5%, 10%, etc.). The procedure 1200 may return to 1226, atwhich the visible light sensor may calculate the lighting level of thesub-area after the intensity level of the selected fixture has beenmathematically increased by an increment. This mathematical increase maybe performed by multiplying the baseline image of the selected fixtureby the dimming level of the fixture. The intensity level of the nextmost influential fixture on the sub-area may be mathematically increaseduntil the calculated lighting level is determined, at 1228, to begreater than or equal to the target lighting level. When the calculatedlighting level is determined to be greater than or equal to the targetlighting level, the visible light sensor may transmit a digital messageto the lighting fixture(s) with control instructions for making thecalculated changes.

At 1234, a determination may be made as to whether the lighting level ofeach sub-area is greater than or equal to the target lighting level. Ifthe lighting level of a sub-area is less than the target lighting level,the lighting level in each sub-area may be calculated at the updatedfixture intensity levels at 1236. As the lighting levels of thepreviously selected fixtures may affect the lighting level of multiplesub-areas, the lighting level in each sub-area may be recalculated afterattempting to increase the lighting level of a single sub-area. Theprocedure 1200 may proceed to 1220 to determine the difference betweenthe updated fixture intensity levels and the target lighting levels ofeach subarea and proceed through steps 1222-1234 until the lightinglevel of each sub-area is greater than or equal to the target lightinglevel. The procedure may then proceed to FIG. 12B.

As illustrated in FIG. 12B, the lighting level in each sub-area may becalculated at the present fixture intensity levels, at 1238. Thelighting level in each sub-area may be analyzed to determine whether thelighting level in any sub-area is greater than an upper-limit lightinglevel. The upper-limit lighting level of a sub-area may be set to thetarget lighting level of the sub-area plus a predefined offset. At 1240,if the lighting level of any sub-area is greater than or equal to thetarget plus a predefined offset, the lighting intensity level of thelighting fixtures may be too bright and may be adjusted to closer to thetarget lighting level. If, at 1240, a determination is made that thelighting level of each sub-area is above the target, but less than thetarget plus the offset, the sensor may transmit one or more digitalmessages to the lighting fixture(s) and/or to the system controller withcontrol instructions at 1256, to control the fixtures to the determinedintensity levels.

If, at 1240, a determination is made that the lighting level of anysub-area is above the target plus the predefined offset, the differencebetween the calculated lighting level of those sub-areas and the targetlighting levels may be determined at 1242. At 1244, the over-illuminatedsub-areas may be ranked from the greatest over-illuminated sub-area tothe least over-illuminated sub-area. The fixture with the highestlighting intensity level affecting the lighting level of the sub-areamay be selected, at 1246, for being adjusted. At 1248, a lower intensitylevel may be determined for the selected fixture. The lower intensitylevel may be the lowest intensity level for the selected fixture thatwill not cause any sub-areas to drop below the target lighting level.The lower intensity level of the selected fixture may be calculated as apercentage based on the illuminance distribution of the fixture for thesub-areas and the known amount of change for each sub-area that iscaused by the percentage of change in the intensity level of theselected fixture. As the adjusted intensity level may cause the fixtureto turn off, a determination may be made at 1250 as to whether thefixture would be turned off. If the lower lighting intensity level ofthe fixture would cause the fixture to turn off, the next highestfixture above the target lighting level with the highest lightingintensity level affecting the lighting level of the sub-area may beselected, at 1252, for being adjusted. The lowest lighting intensitylevel for each of the selected fixtures may continue to be determined,at 1248, until it is determined that the lowest lighting intensity leveldoes not cause the selected fixture to turn off at 1250.

After the lighting intensity level of at least one fixture is adjusted,a determination may be made at 1254 as to whether there are othersub-areas for which the lighting level should be calculated since theadjustment of the fixture. If there are other sub-areas for which thelighting level should be calculated, the procedure 1200 may return to1238 to calculate the lighting level and the sub-areas may continue tobe evaluated to determine whether any sub-area is above the targetlighting level by the offset, at 1240. The offset may be a tolerancelevel above the target lighting level. After each sub-area has beencalculated after the adjustment of one or more fixtures and isdetermined to be above the target lighting level, but below the uppertarget light level, the sensor may transmit one or more digital messagesto the lighting fixture(s) and/or to the system controller with controlinstructions, at 1256, to control the fixtures to the determinedintensity levels.

FIG. 13 shows a flowchart of an example baseline configuration procedure1300 that may be executed by a visible light sensor, such as the visiblelight sensor 180. The baseline configuration procedure 1300 may beexecuted by a control circuit (e.g., the control circuit 310) during aconfiguration procedure for configuring operation during the colorsensor mode of the sensor.

The baseline configuration procedure 1300 may begin at 1310. At 1312,the sensor may set the color temperature to a starting colortemperature. The starting color temperature may be an extreme colortemperature on the color spectrum (e.g., 2,000 Kelvin or 6,500 Kelvin onthe black body curve). The sensor may set the color temperature to astarting color temperature by adjusting one or more lighting controldevices (e.g., lighting fixtures 172, 174, 176, 178).

At 1314, the sensor may record an image. For example, the visible lightsensor may record one or more images that include one or more regions ofinterest within the space. The sensor may, at 1316, store the recordedimage as a baseline image. The color temperature presented within theimage may be set as the baseline color temperature.

The sensor may record the control setting of one or more control devices(e.g., lighting fixtures 172, 174, 176, 178; motorized window treatments150, etc.) that were used to present the baseline color temperature. Atstep 1318, the sensor may associate the control settings with thebaseline color temperature. For example, the sensor may associatelighting fixtures presenting 4,500 Kelvin with the baseline colortemperature.

The sensor may determine, at 1320, if the color temperature is theending color temperature. For example, if the color temperature in theimage is 4,500 Kelvin and the ending color temperature is 4,500 Kelvin,the color temperature and the ending color temperature may be the same.

A similar process may be performed at different color temperatures tostore a baseline image and associated control settings for the differentcolor temperatures. As such, the sensor may begin at a starting colortemperature (e.g., on the black body curve) and incrementally adjust(e.g., by 1 Kelvin, 10 Kelvin, 50 Kelvin, 100 Kelvin, etc.) the colortemperature by a predefined amount (e.g., along the black body curve) torecord and store the baselines and the corresponding control settingsfor each color temperature. If the color temperature and the endingcolor temperature are different, the sensor may move to 1322 andincrementally adjust the color temperature (e.g., by a predefined amountalong the black body curve). For example, the sensor may transmit amessage to one or more lighting control devices to incrementally adjustthe color temperature toward the ending color temperature. Afteradjusting the color temperature, the procedure may move to 1314 andrecord an image at the adjusted light color temperature. The procedure1300 may continue until the ending color temperature is reached.

FIG. 14 shows a flowchart of an example procedure 1400 for controlling acorrelated color temperature (CTT) value based on an image. The colortemperature may be controlled to approximate the color temperature of ablack-body radiator which to human color perception most closely matchesthe light from the sun. The procedure 1400 may be executed by a visiblelight sensor, such as the visible light sensor 180. The procedure 1400may be executed by a control circuit 310 of the sensor (e.g., thecontrol circuit 310) during a color sensor mode of the sensor.

The procedure 1400 may begin at 1410. At 1412, an image of a task areaor other region of interest may be retrieved. A previously capturedimage may be retrieved from memory or the image may be retrieved bycapturing the image with a camera. A color control mask may be appliedto the image at 1414. The color control mask may be applied to disregardportions of the image outside of regions of interest having one or moreknown colors. For example, the color control mask may be applied todisregard the areas outside of a portion of the task area having a knowncolor. The portion of the task area may include a color wheel or anotherobject (e.g., piece of paper, a mobile device of a user, keypads,sensors, etc.) that may be identified in the image or in a predefinedlocation.

The sensor may be configured to determine RGB values of at least onepixel in the image at 1416. A camera on the visible light sensor maytake an R, G, and B reading per pixel. The R, G, and B readings may beincluded in the image data. The sensor may generate high-dynamic-range(HDR) images or lower resolution images that include different types ofimage data for being processed.

In one example, the image may include a color wheel that may beconfigured to display one or more colors. The color wheel may includestandard RGB colors. The sensor may use the color wheel to determine theRGB values. The sensor may be configured to record an image of the colorwheel and measure a color temperature of a light emitted by one or moreof the lighting fixtures (e.g., lighting fixtures 172, 174, 176, 178)using the color wheel. The colors on the color wheel may be identifiedin the reflected light in the generated images of the space. A relativedifference in color temperature from the colors on the color wheel maybe identified in the reflected light captured in the generated images.The color temperature of the light in the space may cause a shift in thecolors detected by the sensor on the color wheel. The sensor maycalculate the shift in the color temperature from the known colors onthe color wheel. This shift may indicate the color temperature of thelight in the space.

At 1418, CIE tristimulus values (XYZ) may be calculated from the RGBvalues. The CIE coordinates (x,y) on the blackbody curve may becalculated, at 1420, from the CIE tristimulus XYZ values. The sensor maymap a sensor response (RGB) to the CIE tristimulus values (XYZ) tocalculate the chromaticity coordinates (x, y) and the CCT. One or moreequations may be used to map the RGB and the XYZ values. For example, Xmay be calculated as (−0.14282)(R)+(1.54924)(G)+(−0.95641)(B); Y (e.g.,luminance) may be calculated as(−0.32466)(R)+(1.57837)(G)+(−0.73191)(B), and/or Z may be calculated as(−0.68202)(R)+(0.77073)(G)+(0.56332)(B). This example may be differentbased on the camera and/or other hardware on the sensor. The colortemperature presented within the room may be based on one or morefactors, including, for example, the presence and/or color of lightemitting diodes (LEDs) presenting a particular color, the age and/oroperability of LED light sources, etc. The x coordinate of the CIEcoordinates (x,y) may be calculated from the CIE tristimulus XYZ valuesusing the formula

$x = {\frac{X}{X + Y + Z}.}$

The y coordinate of the CIE coordinates (x,y) may be calculated from theCIE tristimulus XYZ values using the formula

${y = \frac{Y}{X + Y + Z}}.$

At 1422, McCamy's formula may be used to calculate the CCT from the CIEcoordinates (x,y). The calculated CCT value may be transmitted to thesystem controller or the lighting fixtures for changing the colortemperature of the light emitted by the lighting fixtures. The systemcontroller or the load control device may generate control instructionsfor changing the color temperature based on the calculated CCT value.

FIG. 15 shows a flowchart of an example glare detection and controlprocedure 1500. The glare detection and control procedure 1500 may beexecuted by a sensor, such as the sensor 180. The glare detection andcontrol procedure 1500 may operate during a daylight glare sensor modeof a sensor, such as the sensor 180.

The glare detection and control procedure 1500 may begin at step 1510.At step 1512, the sensor may determine whether a glare condition ispossible. For example, the sensor may determine whether a glarecondition is possible based on the time of day, time of year, locationof the building, direction of the windows, position of the sun in thesky, weather conditions, etc., that may be used to determine theintensity of sunlight at the space.

If the glare condition is determined to be impossible or improbable at1512, the procedure 1500 may end. If the glare condition is determinedto be possible, the procedure 1500 may move to step 1514, and the sensormay record an image. For example, the sensor may record one or moreimages that include one or more regions of interest within a space.

The sensor may analyze the images and identify a lighting intensity inone or more regions of the space at 1516. The sensor may analyze theimages and identify a relative difference of light in the space. Forexample, the sensor may determine whether two or more light intensitiesare being presented within the space. Each lighting intensity may bedetermined by applying a respective mask or baseline to a region ofinterest. The two or more light intensities presented within the spacemay be from different sources or the same source. For example, alighting intensity may relate to sunlight or ambient light beingpresented in space and/or a lighting intensity may relate to artificiallight being presented into the space. The sunlight may enter the spacefrom a window and artificial light may be presented in the spacelighting fixtures, such as lighting fixtures 172, 174, 176, 178. Thesensor may analyze the light intensities and identify whether the lightintensities are sunlight or artificial light. For example, the sensormay identify a lighting intensity as sunlight if the lighting intensityis being presented within a predefined distance from the window. If, at1518, the lighting intensity is artificial light, the procedure 1518 mayend. If the lighting intensity is sunlight provided from the directionof the window, the procedure 1500 may move to 1520.

The sensor may determine if one or more of the light intensities (e.g.,the lighting intensity relating to the sunlight provided by the window)exceeds a sunlight glare threshold, at 1520. The sunlight glarethreshold may be a preferred sunlight glare threshold provided by theuser. The sunlight glare threshold may be a recommended sunlight glarethreshold provided by the lighting manufacturer or lighting designer. Alocalized measure of the lighting intensity (e.g., luminance) mayprovide a measure of the lighting intensity to determine if it is glare.For example, a pixel area measure of the luminance may provide a measureof the lighting intensity to determine if it is glare. If the lightingintensity relating to the sunlight fails to exceed a sunlight glarethreshold, the procedure 1500 may end. If the lighting intensityrelating to the sunlight does exceed a sunlight glare threshold, thesensor may determine one or more regions of interest in which thesunlight reaches. For example, at 1522, the sensor may determine if thesunlight reaches the user task area and/or an area surrounding (e.g.,within a predefined distance of) the user task area (e.g., desk 106,monitor 166, a predefined area around user 192, etc.).

If the sunlight fails to reach at the task area, the procedure 1500 mayend. If, however, the sunlight does reach the task area, the coveringmaterial of each of the motorized window treatments in the space may beadjusted. For example, the sensor may transmit an indication of sunglare at 1524. The sensor may transmit the indication of sunlight glareto the system controller or directly to the motorized window treatments.The system controller may transmit the indication of sunlight glare orcontrol instructions for control instructions for moving the coveringmaterial to the motorized window treatments. The motorized windowtreatments may adjust the covering material in response to theindication or control instructions so that the sunlight fails to reachthe task area, or the area surrounding the task surface (e.g., within apredefined distance thereof). The covering material of each of themotorized window treatments may be adjusted a predefined amount (e.g.,10%, 30%, 60%, 90%, etc.) and/or the covering material of each of themotorized window treatments may be adjusted using the amount of sunlightthat is permitted by the covering material of each of the motorizedwindow treatments. For example, the sensor may continually transmitmessages to adjust the covering material of each of the motorized windowtreatments until the sunlight is identified in the generated images asfailing to reach, or failing to reach a predefined distance from, thetask area. After the window treatment is properly adjusted, theprocedure 1500 may end.

Though steps may be described herein as being performed by the sensor,the sensor may record the images and provide the images to the systemcontroller for performing image analysis, control procedures, and/orother functions described herein.

As described in FIG. 15, the sensor may be configured to identifywhether one or more light intensities are sunlight or artificial light.For example, the sensor may determine that a lighting intensity issunlight based on the lighting intensity being presented within apredefined distance of the window. The sensor may determine that alighting intensity is artificial light based on the lighting intensitybeing presented outside of a predefined distance of the window. Thesensor may, however, determine daylight glare conditions in one or moreadditional ways. For example, the sensor may determine a daylight glarecondition using baseline images captured by the sensor.

FIG. 16 shows a flowchart of an example glare detection and controlprocedure 1600 that may be executed to detect and control glare within aspace (e.g., the room 102). The glare detection and control procedure1600 may be executed by a sensor, such as the sensor 180. The glaredetection and control procedure 1600 may be executed by a controlcircuit of the sensor (e.g., the control circuit 310 during a daylightglare sensor mode of the sensor.

The glare detection and control procedure 1600 may begin at step 1610.The glare detection and control procedure 1600 may determine one or morelight intensities (e.g., sunlight intensity, such as a glare condition)within a space (e.g., the room 102). At 1612, the sensor may determinewhether a glare condition is possible. For example, the sensor maydetermine whether a glare condition is possible based on the time ofday, time of year, location of the building, direction of the windows,position of the sun in the sky, weather conditions, etc., that may beused to determine the intensity of sunlight in the room.

If the glare condition is determined to be impossible or improbable at1612, the procedure 1600 may end. If the glare condition is determinedto be possible, the procedure 1600 may move to 1614, and the sensor mayrecord an image. For example, the sensor may record one or more imagesof one or more regions of interest within the room.

The sensor may analyze the images and identify the lighting intensity(e.g., sunlight intensity, such as a glare condition) within the room,at 1616. The sensor may identify a baseline lighting intensity (e.g.,sunlight intensity, such as a glare condition). A baseline lightingintensity of the room may include recorded images of the room in whichthe amount of sunlight may be zero sunlight, full sunlight, and/or anumber of intervals that may fall between zero sunlight and fullsunlight. For example, the sensor may determine a baseline having zerosunlight by recording an image of the room when sunlight is not present(e.g., at nighttime). The sensor may determine a baseline having zerosunlight by recording an image when the covering material of themotorized window treatments are in a fully closed state. The sensor maydetermine a baseline having full sunlight by recording an image of theroom at a time during the day in which sunlight is predicted to be at afull potential and when the covering material of the motorized windowtreatments are in an open state.

The sensor may be configured to determine baseline intervals (e.g., 10%,20%, 30%, etc.) of sunlight within the room. For example, baselineintervals of sunlight within the room may be provided using one or morepositions of the covering material of the motorized window treatments.Also, or alternatively, baseline intervals of sunlight within the roommay be provided using one or more combinations of environmentalcharacteristics that may affect the presence of sunlight (e.g., the timeof day, the time of year, the weather, the position of the sun, thedirection of the windows, the location of the building, the location ofthe room in the building, etc.). For example, baseline intervals may beprovided in the room at a time of the day in which the sun is predictedto provide full sunlight and/or at which the covering material of themotorized window treatments are opened a predefined amount. The sensormay record one or more image of the room during times of differentsunlight strengths and/or based on the level of the covering material ofthe motorized window treatments being opened to different amounts (e.g.,opened to 10%, 30%, 50%, 70%, or 90% capacity).

At 1618, the sensor may determine whether the lighting intensity withinthe room is greater than a predefined lighting intensity. The predefinedlighting intensity may be a baseline lighting intensity and/or anotherlighting intensity, such as the lighting intensity defined by thesunlight glare threshold. If the lighting intensity within the room isless than or equal to the predefined lighting intensity, the procedure1600 may end. If the lighting intensity within the room is greater thanthe predefined lighting intensity (e.g., the baseline lighting intensityand/or the sunlight glare threshold), the sensor may adjust the coveringmaterial of each of the motorized window treatments, at 1620. The windowtreatment may be adjusted by sending a digital message to the motorizedwindow treatments.

The sensor may adjust the covering material of each of the motorizedwindow treatments so that the lighting intensity (e.g., sunlightintensity) presented within the room is equivalent, or similar, to thelighting intensity of the predefined lighting intensity. The sensor mayadjust the covering material of each of the motorized window treatmentsto the setting of the covering material of each of the motorized windowtreatments at which the predefined (e.g., baseline) lighting intensitywas recorded. The sensor may consider one or more secondaryconsiderations (e.g., time of day, time of year, location of thebuilding, direction of the windows, position of the sun in the sky,weather conditions, etc.) when adjusting the covering material of eachof the motorized window treatments to achieve the baseline lightingintensity.

At 1622, the sensor may determine whether sunlight is presented on orwithin a predefined distance of the task area (e.g., desk 106, monitor166, a predefined area around user 192, etc.). If sunlight is notpresented on or within a predefined distance of the task area, theprocedure 1600 may end. If, however, the sunlight is presented on orwithin a predefined distance of the task area, the covering material ofeach of the motorized window treatments may be adjusted, at 1620. Forexample, the sensor may adjust the covering material of each of themotorized window treatments so that the sunlight is not presented on orwithin a predefined distance of the task area. The covering material ofeach of the motorized window treatments may be adjusted a predefinedamount (e.g., 10%, 30%, 60%, 90%, etc.). The covering material of eachof the motorized window treatments may be adjusted based on the amountof sunlight that is permitted by the covering material of each of themotorized window treatments. For example, if at 1622 sunlight ispresented on or within a predefined distance of the task area, thesensor may continually adjust the covering material of each of themotorized window treatments, at 1620, until the sunlight is notpresented on or within a predefined distance of the task area.

FIG. 17 shows a flowchart of an example configuration procedure 1700that may be executed to configure a sensor (e.g., a visible lightsensor) for operation. The configuration procedure 1700 may be executedusing configuration software, which may be executed on one or moredevices. The configuration procedure 1700 may be executed by a sensor,such as the visible light sensor 180, a system controller, such as thesystem controller 110, and/or a network device, such as the mobiledevice 190.

As shown in FIG. 17, configuration procedure 1700 may begin at 1710. Thesensor may record an image of the space at 1712. At 1714, image data maybe displayed via a graphical user interface (GUI) on the visual displayof a network device, such as the mobile device 190 shown in FIG. 1. Thesensor, the system controller, or the network device may receive therecorded images of the space and generate the image data from therecorded images. Image data may be associated with pixel data (e.g.,having a red value, a green value, and a blue value). For example, theimage data may be processed to indicate objects identified in the imagedata. The network device may receive a user selection of a controlstrategy at 1716. The control strategy may be configured for performingcontrol (e.g., generating control instruction) of one or more loadcontrol devices based on detected environmental characteristics when thesensor is operating in a sensor mode. For example, the control strategymay be executed to control one or more load control devices in responseto detection of a daylight sensor event, daylight glare sensor event,occupancy/vacancy sensor event, color temperature sensor event, lightingintensity, or other environmental characteristics within the spaceduring a sensor mode.

The network device may receive control parameters for the selectedcontrol strategy at 1718. For example, the network device may receiveuser selections that indicate the sensitivity for detecting anoccupancy/vacancy sensor event during an occupancy/vacancy sensor mode(e.g., high, medium, or low), a timeout for a vacancy timer, targetlighting level levels, types of target lighting level control (e.g.,uniform vs. gradient as described in FIGS. 12A and 12B), a target coloror color temperature, daylight penetration distance and/or bufferdistance (e.g., distance from task surface that daylight penetration isprevented from exceeding), and/or other control parameters forperforming control of the sensor and/or the one or more load controldevices. The control parameters may also, or alternatively, includepreferred lighting intensity parameters, daylighting glare parameters,color temperature parameters, etc.

The network device may receive an indication of user defined regions ofinterest or disinterest for the selected control strategy at 1720. Thenetwork device may receive user-selected regions of interest that are tobe unmasked or regions of disinterest that are to be masked for thecontrol strategy. For example, the user-selected regions of interest foroccupancy/vacancy control may include the area around a user's taskarea, a user's chair, a user's keyboard, etc. The user-selected regionsof disinterest for occupancy/vacancy control may include the area arounda doorway or windows, computer monitors, television screens, or otherareas of disinterest so that any movement detected within these areas ofdisinterest does not affect the occupancy/vacancy control of loadcontrol devices.

The user may define, via a network device (e.g., the mobile device 190),the control strategy. For example, the user may identify environmentalcharacteristics that may be detected for performing load controlaccording to one or more control strategies. The network device may lista number of control strategies, such as, general room occupancy sensingcontrol, keyboard occupancy sensing control, task surface lighting levelcontrol, or sunlight penetration control. The network device maydetermine (e.g., automatically determine), based on the user'sidentification, a sensor mode (e.g., occupancy sensor mode), and/orcontrol parameters (e.g., high sensitivity) for the identified controlstrategy.

At 1722, a determination may be made as to whether the configuration ofcontrol strategies is complete. For example, the user may be askedthrough the configuration software on the network device whether theyare done performing configuration. If the configuration is incomplete,the configuration procedure 1700 may return to 1716 and the user mayselect the control strategy for being configured. If the configurationis determined to be complete, at 1722, configuration data may betransmitted to the appropriate devices at 1724. The configuration datamay include the control parameters and/or the regions ofinterest/disinterest for the control strategy. The configuration datamay be transmitted to the sensor upon which control instructions may begenerated in response to images recorded by the sensor. The sensor maybe configured, at 1726, for performing in accordance with theconfiguration data during normal operation. For example, the sensor maybe configured with the selected control parameters and/or the userdefined masks when operating to perform load control based on theselected control strategy.

As the sensor may be installed at a location at which the sensor canrecord an image of a space, the sensor may operate in a manner thatprotects the privacy of the users in the space. For example, a sensormay be configured to protect the privacy of the users of a space viasoftware, a removable module, a special sensor, and/or communication ondifferent communication links during configuration and operation of theload control system. The configuration software that may be implementedduring the procedure 1700 may be used in a way that protects the privacyof the users of a space. The configuration software may case the sensorto communicate on a wired communication link, or a different wirelesscommunication link than the sensor may operate during operation. Theconfiguration software may be uninstalled from the sensor whenconfiguration of the sensor is complete, such that the sensor may leaveconfiguration mode and move to operation modes of the sensor foridentifying images and transmitting messages for load control when theconfiguration of the sensor is complete. During operation of the sensor,operation software may be installed by the sensor. The operationsoftware may include the operation modes for identifying images andtransmitting messages for load control. The operation software mayprevent the transmission of actual images or other image data that maybe transmitted from the sensor when the configuration software isinstalled. If the sensor is capable of transmitting images or otherimage data during operation, the sensor may use a wired or wirelesscommunication link that is different than the communication link usedfor configuration.

During configuration of the sensor, a configuration module may becoupled to (e.g., installed in) the sensor that allows the sensor totransmit images or other image data. When the configuration module isinstalled in the sensor, the control circuit 310 (shown in FIG. 3) maytransmit an image recorded by a camera, such as the camera 322, or otherimage data via a communication link. The module may have wired and/orwireless capabilities. The sensor may include a communication circuitfor transmitting and/or receiving the RF signals 108 (e.g., using theproprietary protocol). The configuration module may include acommunication circuit for transmitting and/or receiving the RF signals109 (e.g., using the standard protocol). When the configuration moduleis installed in the sensor and the communication circuit of theconfiguration module is electrically coupled to the sensor, the sensormay record an image of the space and transmit the image or other imagedata to the system controller or the network device. The network deviceor the system controller may transmit the configuration data to thesensor while the configuration module is installed in the sensor, andthe sensor may store the configuration data in memory. After theconfiguration of the sensor (e.g., during operation of the sensor forload control), the configuration module may be removed from the sensor,resulting in the sensor being unable to transmit images or other imagedata. With the configuration module removed, the sensor may be unable totransmit images or image data. If the sensor is capable of transmittingimages or other image data during operation, the sensor may use a wiredor wireless communication link that is different than the communicationlink used for configuration.

Another way to protect the privacy of users may be to use a specialconfiguration sensor. The configuration sensor may be installed on, orin the same location as, the sensor and may transmit images of the room.The configuration sensor may have a structure that is identical, orsimilar, to the sensor. The configuration sensor may be configured torecord an image using a camera. The configuration sensor may transmitimage data (e.g., an image or other image data) to the system controllerand/or the network device. The configuration sensor may communicate on adifferent communication link than the sensor. The configuration dataresulting from the image data may be transmitted to the sensor. Theconfiguration sensor may be uninstalled after configuration of thesensor. For example, the sensor may leave the configuration mode andmove to operation modes of the sensor when the configuration of thesensor is complete. The visible light sensor may be installed in placeof the configuration sensor for use during operation using theconfiguration data generated from the images or other image data fromthe space. The visible light sensor may be incapable of transmittingimages or other image data after being installed in place of, or afterthe removal of, the configuration sensor. If the visible light sensor iscapable of transmitting images or other image data during operation, thevisible light sensor may use a wired or wireless communication link thatis different than the communication link used for configuration.

FIG. 18 shows a flowchart of an example configuration procedure 1800that may be executed to configure a sensor (e.g., a visible lightsensor) for operation. The configuration procedure 1800 may be executedusing configuration software, which may be executed on one or moredevices. The configuration procedure 1800 may be executed by a visiblelight sensor, such as the visible light sensor 180 or 300, a systemcontroller, such as the system controller 110, and/or a network device,such as the mobile device 190.

As shown in FIG. 18, the configuration procedure 1800 may begin at 1810.At 1812, an application may be opened on the network device. A roomidentifier may be entered and received by the network device at 1814.For example, the user may enter a room identifier, such as an identifierof the living room, a conference room, a hotel room, etc. As a roomidentifier is being stored for the configuration of the space in theroom, the configuration of a space having a given room identifier may beused as a template (e.g., a configuration template) for configuring thevisible light sensor and/or load control within a similar space. Aconfiguration template may be copied and applied to other spaces forperforming load control. The configuration template may include similarmasks, regions of interest, control strategies, etc.

The network device may receive a user selection of a control strategy at1816. The control strategy may be defined for a sensor mode forcontrolling one or more load control devices in response to detection ofone or more sensor events. For example, the control strategy may includethe control of one or more load control devices in response to daylight,daylight glare, occupancy/vacancy, color temperature, lightingintensity, or other environmental characteristics within the space.

The network device may receive control parameters for the selectedcontrol strategy at 1818. For example, the network device may receiveuser selections that indicate the sensitivity for detecting anoccupancy/vacancy condition during occupancy sensing (e.g., high,medium, or low), a timeout for a vacancy timer, and/or other controlparameters for performing control of the visible light sensor and/or theone or more load control devices. The control parameters may also, oralternatively, include preferred lighting intensity parameters,daylighting glare parameters, color temperature parameters, etc.

The network device may receive a selection, at 1820, from a user of anobject type that is being identified for configuration. For example, theobject type may be a user task area (e.g., a desk), a door, a window, oranother object type for being identified in an image during operation ofa visible light sensor. The network device may transmit theconfiguration data at 1822, which may include the control strategy,control parameters, and/or selected object type for being defined. Theconfiguration data may be transmitted to the visible light sensor.

The network device may be used to identify objects within the space. Theobjects that are identified may be used for masking areas of the spaceor detecting other environmental characteristics within the space forperforming load control. At 1824, a determination may be made as towhether the user will define the boundary of the selected object type bytracing the boundary of the object. If the user is not tracing theboundary of the object within the image, the user may otherwise definethe object of the selected type within the room. For example, at 1826,the user may place a network device (e.g., mobile phone, tablet, etc.)or other predefined object identifier on the object of the selected typewithin the space. The visible light sensor may record an image of thenetwork device on the object of the selected type within the space. At1828, the visible light sensor may determine and store the boundaries ofthe object on which the network device is located within the image. Thenetwork device may be a predefined object identifier, as it may bestored in memory at the visible light sensor as the object used toidentify other objects in the images. The boundaries of the objects inan image may be determined by locating the boundaries of the nextlargest object on which the network device resides within the image(e.g., for a predefined period of time). Though a network device may bedescribed as being used to identify the boundaries of the selectedobject type, another type of object may similarly be identified withinan image and used to identify the boundaries of another object withinthe image.

If the user is to trace the boundary of the object within the image, at1824, the user may trace the boundaries of the object with anidentifying device (e.g., a finger, a laser pointer, a network device,etc.). The visible light sensor may record images of the user tracingthe boundaries of the object with the identifying device in the spaceand recognize the boundaries of the object being traced within theimages. For example, the user may trace the edges of a task area (e.g.,desk) or doorway with the user's network device (e.g., mobile phone),which may be recognized by the visible light sensor as the boundary ofthe object having the selected type. The boundaries of the object may bestored at 1832.

At 1834, a determination may be made as to whether the configuration inprocedure 1800 is complete. For example, the user may be asked throughthe configuration software on the network device whether they are doneidentifying objects for being identified for the selected controlstrategies. If the configuration in procedure 1800 is incomplete, theconfiguration procedure 1800 may return to 1816 and the user may selectthe control strategy for being configured. If the configuration inprocedure 1800 is determined to be complete, at 1834, the visible lightsensor may be configured for normal operation at 1836. For example,configuration data including the boundaries of the objects defined inthe configuration procedure 1800. The boundaries of the defined objectsin the space may be used by the visible light sensor to define masks oridentify environmental characteristics for performing load control.

Indications of user selections may be transmitted to the visible lightsensor, upon which the visible light sensor may be configured foroperation in response to images recorded thereon. During configuration,the visible light sensor may be prevented from transmitting the imageson which configuration is performed. This may protect the privacy of theoccupants within the space.

FIG. 19 shows a flowchart of an example configuration procedure 1900that may be executed to automatically configure a sensor (e.g., avisible light sensor) for operation. The autoconfiguration procedure1900 may be executed using configuration software, which may be executedon one or more devices. The autoconfiguration procedure 1900 may beexecuted by a visible light sensor, such as the visible light sensor 180or 300, a system controller, such as the system controller 110, and/or anetwork device, such as the mobile device 190.

As shown in FIG. 19, the configuration procedure 1900 may begin at 1910.At 1912, an image may be recorded by the visible light sensor. The imagemay be processed by the visible light sensor, at 1914, to discoverobjects. The objects may be predefined in size and/or shape. The objectsmay have been previously defined (e.g., using the configurationprocedure 1800 in FIG. 18, or otherwise defined by the system). Objectsmay have been moved within the space and images may be processed toidentify objects having the same boundaries in a changed location. Theimage may be processed to identify the existence of previously absentobjects, or the absence of previously existing objects.

At 1916, the image may be processed to determine a location and type ofan object in the space. A control strategy and control parameters may bedetermined, at 1918, based on the determined object type. For example,the identification of a desk may indicate daylighting or glare controlbased on the location of the user task area, windows may indicate glarecontrol, and/or other predefined objects may be identified to indicateother types of control strategies. At 1920, a region of interest ordisinterest may be identified using the location and type of the object.For example, when a desk is identified, a mask may be applied to therest of the room, or outside a predefined distance of the desk, todefine a region of interest for detecting daylight or daylight glare.Multiple masks may be applied to the same object if multiple controlstrategies are determined for the same object. For example, when an areawithin or around a desk is used for detecting occupancy/vacancy, a maskmay be applied to the rest of the room, outside a predefined distance ofthe desk, or within a predefined space on the desk, to define a regionof interest for detecting occupancy/vacancy.

A determination may be made at 1922 as to whether a mask already existsfor the identified object. If a mask does not already exist for theobject, a mask may be created at 1924 based on the identified locationof the object in the space. If a mask already exists for the object, theexisting mask may be updated at 1926 based on the identified location ofthe object (e.g., change in location of an object).

The configuration data may be stored at 1928. The configuration data mayinclude the control strategy, the control parameters, the object type,the object location, the regions of interest/disinterest, any masks,and/or other configuration data. A determination may be made as towhether additional control options are to be determined for the objectat 1930. For example, if there is a task surface identified in theimage, the visible light sensor may auto configure the occupancy/vacancysensing operation around the task surface. If, at 1930, there are morecontrol options that are determined for being configured at the tasksurface, the visible light sensor may auto configure those other controloptions (e.g., daylighting operations, daylight glare control to preventglare on the task surface, etc.). The configuration procedure may have anumber of control options stored for a given object that is identifiedin the image. If no additional control options are to be determined forthe object, a determination may be made, at 1932, as to whether moreobjects have been discovered in the image. If other objects have beendiscovered, the procedure 1900 may return to 1916 to process the imageto determine the location and the type of the next object. Otherwise,the procedure 1900 may end.

FIG. 20 shows a flowchart of an example zone configuration procedure2000 that may be executed to configure one or more zones within a space.The zone configuration procedure 2000 may be executed usingconfiguration software, which may be executed on one or more devices.The zone configuration procedure 2000 may be executed by a sensor (e.g.,a visible light sensor, such as the visible light sensor 180) a systemcontroller (e.g., the system controller 110), and/or a network device(e.g., the mobile device 190.

As shown in FIG. 20, the zone configuration procedure 2000 may begin at2010. At 2012, an image may be recorded by the visible light sensor. Theimage may be sent to the system controller for processing, or processedlocally by the visible light sensor. The image may be processed, at2014, to discover objects. Nighttime images may be processed, at 2016,to identify lighting types and locations. The lighting types may includefunctional lights and/or decorative lights. The functional lights may bedownlights. The decorative lights may be wallwash lights, wall sconces,and/or other decorative lights. The lighting types may be identified bythe location of the light source, the location of the light output inthe space, and/or the lighting pattern being output by the light source

At 2018, a determination may be made as to whether the lighting isdecorative lighting. If the lighting fixtures include decorativelighting fixtures at 2018, the decorative lighting fixtures may begrouped by location. For example, if there are multiple decorativelights along one wall and multiple decorative lights along another wall,the decorative lights on each wall may be grouped into two differentzones that each include the decorative lights located on the respectivewall. In another example, decorative lights in a cove within the roommay be included in a separate group from other decorative lights in theroom, such as wallwashes. The grouping may include the decorativelighting fixtures on the same wall, the decorative lighting fixtures inthe space, the decorative lighting fixtures within a predefined distanceof one another, etc. A lighting zone may be created for controlling thedecorative lighting fixtures in each group at 2022. The lighting zonemay be included in configuration data that is stored at the visiblelight sensor for performing load control.

At 2024, a determination may be made as to whether a window isidentified in the space. If a window is identified in the space at 2024,lighting fixtures (e.g., functional and/or decorative lighting fixtures)may be identified at 2026 that are within a predetermined distance fromthe window. The lighting fixtures that are within the predetermineddistance from the window may be included in a daylighting zone that iscreated at 2028. The daylighting zone may be included in configurationdata that is stored at the visible light sensor for performing loadcontrol.

At 2030, a determination may be made as to whether a presentation areais identified in the space. If a presentation area is identified in thespace at 2030, lighting fixtures (e.g., functional and/or decorativelighting fixtures) may be identified at 2032 that illuminate thepresentation area. The lighting fixtures that identify the presentationarea may be included in a controlled zone that is created at 2034. Thecontrolled zone may be included in configuration data that is stored atthe visible light sensor for performing load control.

FIG. 21 is a block diagram illustrating an example network device 2100as described herein. The network device 1800 may be a mobile device 190,as shown in FIG. 1, for example. The network device 2100 may include acontrol circuit 2102 for controlling the functionality of the networkdevice 2100. The control circuit 2102 may include one or more generalpurpose processors, special purpose processors, conventional processors,digital signal processors (DSPs), microprocessors, integrated circuits,a programmable logic device (PLD), application specific integratedcircuits (ASICs), or the like. The control circuit 2102 may performsignal coding, data processing, power control, image processing,input/output processing, and/or any other functionality that enables thenetwork device 2100 to perform as described herein.

The control circuit 2102 may store information in and/or retrieveinformation from the memory 2104. The memory 2104 may include anon-removable memory and/or a removable memory. The non-removable memorymay include random-access memory (RAM), read-only memory (ROM), a harddisk, or any other type of non-removable memory storage. The removablememory may include a subscriber identity module (SIM) card, a memorystick, a memory card, or any other type of removable memory.

The network device 2100 may include a communications circuit 2108 fortransmitting and/or receiving information. The communications circuit2108 may perform wireless and/or wired communications. Thecommunications circuit 2108 may include an RF transceiver or othercircuit capable of performing wireless communications via an antenna.Communications circuit 2108 may be in communication with control circuit2102 for transmitting and/or receiving information.

The control circuit 2102 may also be in communication with a display2106 for providing information to a user. The processor 2102 and/or thedisplay 2106 may generate GUIs for being displayed on the network device2100. The display 2106 and the control circuit 2102 may be in two-waycommunication, as the display 2106 may include a touch screen modulecapable of receiving information from a user and providing suchinformation to the control circuit 2102. The network device 2100 mayalso include an actuator 2112 (e.g., one or more buttons) that may beactuated by a user to communicate user selections to the control circuit2102.

Each of the modules within the network device 2100 may be powered by apower source 2110. The power source 2110 may include an AC power supplyor DC power supply, for example. The power source 2110 may generate asupply voltage VCC for powering the modules within the network device2100.

FIG. 22 is a block diagram illustrating an example system controller2200 as described herein. The system controller 2200 may include acontrol circuit 2202 for controlling the functionality of the systemcontroller 2200. The control circuit 2202 may include one or moregeneral purpose processors, special purpose processors, conventionalprocessors, digital signal processors (DSPs), microprocessors,integrated circuits, a programmable logic device (PLD), applicationspecific integrated circuits (ASICs), or the like. The control circuit2202 may perform signal coding, data processing, power control, imageprocessing, input/output processing, or any other functionality thatenables the system controller 2200 to perform as described herein. Thecontrol circuit 2202 may store information in and/or retrieveinformation from the memory 2204. The memory 2204 may include anon-removable memory and/or a removable memory. The non-removable memorymay include random-access memory (RAM), read-only memory (ROM), a harddisk, or any other type of non-removable memory storage. The removablememory may include a subscriber identity module (SIM) card, a memorystick, a memory card, or any other type of removable memory.

The system controller 2200 may include a communications circuit 2206 fortransmitting and/or receiving information. The communications circuit2206 may perform wireless and/or wired communications. The systemcontroller 2200 may also, or alternatively, include a communicationscircuit 2208 for transmitting and/or receiving information. Thecommunications circuit 2206 may perform wireless and/or wiredcommunications. Communications circuits 206 and 2208 may be incommunication with control circuit 2202. The communications circuits2206 and 2208 may include RF transceivers or other communicationsmodules capable of performing wireless communications via an antenna.The communications circuit 2206 and communications circuit 2208 may becapable of performing communications via the same communication channelsor different communication channels. For example, the communicationscircuit 2206 may be capable of communicating (e.g., with a networkdevice, over a network, etc.) via a wireless communication channel(e.g., BLUETOOTH®, near field communication (NFC), WIFI®, WI-MAX®,cellular, etc.) and the communications circuit 2208 may be capable ofcommunicating (e.g., with control devices and/or other devices in theload control system) via another wireless communication channel (e.g.,WI-FI® or a proprietary communication channel, such as CLEAR CONNECT™).

The control circuit 2202 may be in communication with an LED indicator2212 for providing indications to a user. The control circuit 2202 maybe in communication with an actuator 2214 (e.g., one or more buttons)that may be actuated by a user to communicate user selections to thecontrol circuit 2202. For example, the actuator 2214 may be actuated toput the control circuit 2202 in an association mode and/or communicateassociation messages from the system controller 2200.

Each of the modules within the system controller 2200 may be powered bya power source 2210. The power source 2210 may include an AC powersupply or DC power supply, for example. The power source 2210 maygenerate a supply voltage VCC for powering the modules within the systemcontroller 2200.

FIG. 23 is a block diagram illustrating an example control-targetdevice, e.g., a load control device 2300, as described herein. The loadcontrol device 2300 may be a dimmer switch, an electronic switch, anelectronic ballast for lamps, an LED driver for LED light sources, an ACplug-in load control device, a temperature control device (e.g., athermostat), a motor drive unit for a motorized window treatment, orother load control device. The load control device 2300 may include acommunications circuit 2302. The communications circuit 2302 may includea receiver, an RF transceiver, or other communications module capable ofperforming wired and/or wireless communications via communications link2310. The communications circuit 2302 may be in communication withcontrol circuit 2304. The control circuit 2304 may include one or moregeneral purpose processors, special purpose processors, conventionalprocessors, digital signal processors (DSPs), microprocessors,integrated circuits, a programmable logic device (PLD), applicationspecific integrated circuits (ASICs), or the like. The control circuit2304 may perform signal coding, data processing, power control,input/output processing, or any other functionality that enables theload control device 2300 to perform as described herein.

The control circuit 2304 may store information in and/or retrieveinformation from the memory 2306. For example, the memory 2306 maymaintain a registry of associated control devices and/or controlconfiguration instructions. The memory 2306 may include a non-removablememory and/or a removable memory. The load control circuit 2308 mayreceive instructions from the control circuit 2304 and may control theelectrical load 2316 using the received instructions. The load controlcircuit 2308 may send status feedback to the control circuit 2304regarding the status of the electrical load 2316. The load controlcircuit 2308 may receive power via the hot connection 2312 and theneutral connection 2314 and may provide an amount of power to theelectrical load 2316. The electrical load 2316 may include any type ofelectrical load.

The control circuit 2304 may be in communication with an actuator 2318(e.g., one or more buttons) that may be actuated by a user tocommunicate user selections to the control circuit 2304. For example,the actuator 2318 may be actuated to put the control circuit 2304 in anassociation mode and/or communicate association messages from the loadcontrol device 2300.

Although features and elements are described herein in particularcombinations, each feature or element can be used alone or in anycombination with the other features and elements. The methods describedherein may be implemented in a computer program, software, or firmwareincorporated in a computer-readable medium for execution by a computeror processor. Examples of computer-readable media include electronicsignals (transmitted over wired or wireless connections) andcomputer-readable storage media. Examples of computer-readable storagemedia include, but are not limited to, a read only memory (ROM), arandom access memory (RAM), removable disks, and optical media such asCD-ROM disks, and digital versatile disks (DVDs).

What is claimed is:
 1. A device for performing control based on imagesof a space, the device comprising: an image recording circuit configuredto capture images of the space; and a control circuit configured to:retrieve, from the image recording circuit, an image of the space;process the image to determine a location of at least one object withinthe space; based on the location of the at least one object within thespace, automatically configure a first mask to identify a first unmaskedportion of the image of the space; and based on the location of the atleast one object within the space, enable a load control device toperform control of one or more electrical loads in the space.
 2. Thedevice of claim 1, wherein the control circuit being configured toenable the load control device to perform control of the one or moreelectrical loads in the space comprises the control circuit beingconfigured to: based on the location of the at least one object withinthe space, generate control instructions for performing control of theone or more electrical loads in the space.
 3. The device of claim 2,further comprising a communications circuit, wherein the control circuitis further configured to transmit, via the communications circuit, thegenerated control instructions for performing control of the one or moreelectrical loads in the space.
 4. The device of claim 3, wherein thegenerated control instructions are transmitted to a system controller.5. The device of claim 3, wherein the generated control instructions aretransmitted to the load control device.
 6. The device of claim 1,wherein the control circuit being configured to generate controlinstructions for control for performing control comprises the controlcircuit being further configured to process the first unmasked portionof the image of the space to sense an environmental characteristic ofthe space.
 7. The device of claim 1, wherein the control circuit beingconfigured to generate control instructions for control for performingcontrol comprises the control circuit being further configured toautomatically create a lighting zone comprising a plurality of lightingcontrol devices configured to control respective lighting loads thataffect lighting in the location of the at least one object, wherein thelighting zone comprising the plurality of lighting control devices areconfigured to be controlled collectively.
 8. A method comprising:retrieving an image of a space; processing the image to determine alocation of at least one object within the space; based on the locationof the at least one object within the space, automatically configuring afirst mask to identify a first unmasked portion of the image of thespace; and based on the location of the at least one object within thespace, enabling a load control device to perform control of one or moreelectrical loads in the space.
 9. The method of claim 8, furthercomprising generating control instructions for performing control of theone or more electrical loads in the space based on the location of theat least one object within the space.
 10. The method of claim 9, furthercomprising transmitting the generated control instructions forperforming control of the one or more electrical loads in the space. 11.The method of claim 10, wherein the generated control instructions aretransmitted to a system controller.
 12. The method of claim 10, whereinthe generated control instructions are transmitted to the load controldevice.
 13. The method of claim 8, further comprising processing thefirst unmasked portion of the image of the space to sense anenvironmental characteristic of the space.
 14. The method of claim 8,further comprising automatically creating a lighting zone, the lightingzone comprising a plurality of lighting control devices configured tocontrol respective lighting loads that affect lighting in the locationof the at least one object, wherein the lighting zone comprising theplurality of lighting control devices are configured to be controlledcollectively.
 15. A computer-readable storage medium having computerexecutable instructions stored thereon that, when executed by a controlcircuit, cause the control circuit to: retrieve an image of a space;process the image to determine a location of at least one object withinthe space; automatically configure a first mask to identify a firstunmasked portion of the image of the space based on the location of theat least one object within the space; and enable a load control deviceto perform control of one or more electrical loads in the space based onthe location of the at least one object within the space.
 16. Thecomputer-readable storage medium of claim 15, wherein the instructions,when executed, further cause the control circuit to generate controlinstructions for performing control of the one or more electrical loadsin the space based on the location of the at least one object within thespace.
 17. The computer-readable storage medium of claim 16, wherein theinstructions, when executed, further cause the control circuit totransmit the generated control instructions for performing control ofthe one or more electrical loads in the space.
 18. The computer-readablestorage medium of claim 17, wherein the generated control instructionsare transmitted to at least one of a system controller or the loadcontrol device.
 19. The computer-readable storage medium of claim 15,wherein the instructions, when executed, further cause the controlcircuit to process the first unmasked portion of the image of the spaceto sense an environmental characteristic of the space.
 20. Thecomputer-readable storage medium of claim 15, wherein the instructions,when executed, further cause the control circuit to automatically createa lighting zone, the lighting zone comprising a plurality of lightingcontrol devices configured to control respective lighting loads thataffect lighting in the location of the at least one object, wherein thelighting zone comprising the plurality of lighting control devices areconfigured to be controlled collectively.